Structure-from-Motion 3D Reconstruction of the Historical Overpass Ponte della Cerra: A Comparison between MicMac^® Open Source Software and Metashape^®

Abstract

In recent years, the performance of free-and-open-source software (FOSS) for image processing has significantly increased. This trend, as well as technological advancements in the unmanned aerial vehicle (UAV) industry, have opened blue skies for both researchers and surveyors. In this study, we aimed to assess the quality of the sparse point cloud obtained with a consumer UAV and a FOSS. To achieve this goal, we also process the same image dataset with a commercial software package using its results as a term of comparison. Various analyses were conducted, such as the image residuals analysis, the statistical analysis of GCPs and CPs errors, the relative accuracy assessment, and the Cloud-to-Cloud distance comparison. A support survey was conducted to measure 16 markers identified on the object. In particular, 12 of these were used as ground control points to scale the 3D model, while the remaining 4 were used as check points to assess the quality of the scaling procedure by examining the residuals. Results indicate that the sparse clouds obtained are comparable. MicMac^® has mean image residuals equal to 0.770 pixels while for Metashape^® is 0.735 pixels. In addition, the 3D errors on control points are similar: the mean 3D error for MicMac^® is equal to 0.037 m with a standard deviation of 0.017 m, whereas for Metashape^®, it is 0.031 m with a standard deviation equal to 0.015 m. The present work represents a preliminary study: a comparison between software packages is something hard to achieve, given the secrecy of the commercial software and the theoretical differences between the approaches. This case study analyzes an object with extremely complex geometry; it is placed in an urban canyon where the GNSS support can not be exploited. In addition, the scenario changes continuously due to the vehicular traffic.

1. Introduction

In recent years, technological advancements in the unmanned aerial vehicle (UAV) industry have drastically transformed survey techniques for 3D model reconstruction. These improvements exploit the evolution of algorithms from computer vision, which once was considered the bottleneck of such techniques. Now, they require less time and are mostly automated.

Moreover, the performance of free-and-open-source software (FOSS) for image processing is increasing, allowing users to conduct the entire photogrammetric process without being obliged to purchase an expensive software license. These two crucial trends present opportunities for both researchers and surveyors. Indeed, with consumer UAVs and FOSS, one can potentially survey an object at a considerably lower cost compared with the past, when UAVs were available to only a few, and for professional results, one had to purchase expensive software licenses. The current tendency in the photogrammetric community is to employ an increased number of images to ensure high overlap values. Today, several photogrammetric software solutions are available, both commercial and FOSS.

Table 1. Overview of available photogrammetric software packages.

Name	OS	Pricing
3DFlow Zephyr [4]	Windows	360 EUR/month
Autodesk Recap [5]	Windows	55 EUR/month
Agisoft Metashape [6]	Windows, macOS, Linux	4075 EUR
BAE Systems SOCET GXP [7]	Windows	On request
Bentley ContextCapture [8]	Windows	from 211 EUR/month
ColMap [9]	Windows, macOS, Linux	Free
Drone Deploy [10]	Windows, macOS, Android, iOS	299 EUR/month
Planetek IMAGINE [11]	Windows	On request
Meshroom [12]	Windows, Linux	Free
MicMac [13]	Windows, macOS, Linux	Free
Multi-view environment [14]	Windows, macOS	Free
Photometrix IWitness Pro [15]	Windows	986 EUR
PhotoModeler [16]	Windows	from 50 EUR/month
Pix4D Mapper [17]	Windows, macOS, Android, iOS	from 185 EUR/month
PMS AG Elcovision 10 [18]	Windows	On request
OpenDroneMap WebODM [19]	Windows, macOS	from 50 EUR
OpenMVG [20]	Windows, macOS, Linux	Free
RealityCapture [21]	Windows	3220 EUR
SimActive Correlator 3D [22]	Windows	from 250 EUR/month
Regard3D [23]	Windows, macOS, Linux	Free
Trimble InPho [24]	Windows	On request
VisualSFM [25]	Windows, macOS, Linux	Free

presents an overview of the available photogrammetric software packages. It was demonstrated that Structure-from-Motion (SfM) algorithms perform well in many applications. At this time, SfM 3D surveys became a viable cost-effective alternative surveying method to aerial LiDAR [1]. In addition, 3D photogrammetry has improved in terms of point density and geometric accuracy through increased overlap between images, improved radiometry, and significant progress in multi-view matching; moreover, the graphics processing unit (GPU) computation power has increased [2] continuously; regardless, the advantages and disadvantages of photogrammetry and LiDAR compensate for each other. Horizontal errors in photogrammetry are usually smaller than those in LiDAR, whereas LiDAR can obtain higher vertical than horizontal accuracy [3].

In the past decade, several photogrammetric applications employing UAVs have been documented. UAVs have been efficiently employed used in various fields, including geomorphological analyses [26,27,28], hydraulics modelling [29], agriculture and forest analyses [30,31,32,33], emergency management support [34,35,36], infrastructure monitoring [37,38,39], and cultural heritage monitoring and 3D reconstruction [40,41,42,43]. The literature also provides numerous examples of combined photogrammetric and LiDAR surveys [44,45,46].

This research aims to determine if MicMac^® can be employed for 3D complex object reconstruction in non-optimal survey conditions, obtaining results comparable with a commercial software solution. MicMac^® has been employed by various researchers both for aerial and terrestrial photogrammetry. Griffiths and Burningham [47] compared PhotoScan^® and MicMac^® concluding that the latter can generate significantly more accurate Digital Surface Models (DSMs) because a more complex lens distortion model is included. Altman et al. [48] tested MicMac^® for terrestrial photogrammetry highlighting that feature extraction results from MicMac^® are comparable with those from Photoscan^®. It is possible to produce complete models in MicMac^® as well, but it likely requires a more refined image set. Jaud et al. [49] compared the DSM obtained by MicMac^® and Photoscan^® concluding that both software can provide satisfying results; nonetheless Photoscan^® is more straightforward to use but its source code is not open, whereas MicMac^® is recommended for experimented users as it is more flexible. In the present work, we focused our attention on the capabilities of MicMac^® to reconstruct a complex object with aerial imagery obtained with consumer UAV. For comparison, the same dataset was processed by either the FOSS and the commercial software package. The qualities of the generated tie point (TP) clouds were analyzed by evaluating the results of the bundle block adjustment (BBA) examining the image residuals and the 3D errors on ground control points (GCPs). Subsequently, the 3D errors on check points (CPs), which are not included in the BBA, are investigated. Additionally, the quality of the results was assessed based on the statistical analysis of the Cloud-to-Cloud distances and the relative accuracy.

The remainder of this paper is organized as follows: Section 2 reports a brief introduction to photogrammetry highlighting the differences and similarities of approach by the two software packages investigated. Section 3 presents the case study of the Ponte della Cerra overpass, and Section 4 provides the comparison of the results of the reconstructed models obtained with and a Agisoft Metashape^®. Section 5 draws conclusions, including a discussion of the results as well as future developments. Lastly, Appendix A and Appendix B define the workflow of the FOSS and the commercial software package, respectively.

2. Material and Methods

Nowadays, SfM photogrammetry has mainly replaced traditional photogrammetry. The principles of Sfm photogrammetry can be found in [50]. Figure 1 depicts the main stages of this process along with the identification of the roles in the investigated software packages, namely Agisoft Metashape^® Pro and MicMac^®. Regarding the acquisition scheme, it is known that image acquisition depends primarily on the type of object. For approximately planar objects the parallel method is useful. In this acquisition scheme, none of the images will cover the entire scene; instead, every image is a tile of the scene. The present work is about the 3D reconstruction of a historical overpass; given the complexity of the object, it was split into three separate parts, namely the north facade, the south facade, and the extrados. When surveying each facade also images capturing the corresponding side of the intrados were collected for a total number of collected images equal to 222. The facades, as well as the extrados, were treated as planar objects taking horizontal and vertical images, respectively. Then, to improve object reconstruction, to reduce shadow areas, and to include some common elements for projects merging, 52 oblique images were collected. Lastly, one of the most important parameters to consider when planning a survey using the parallel method is the overlap between images. Two different overlaps must be considered: the along-track overlap, and the cross-track overlap. In the present survey, the mean along-overlap was equal to 80%, while the cross-track was equal to 55%.

2.1. MicMac^® Photogrammetric Processing

The photogrammetric process (TP search, estimation of camera poses, densification) is conducted using MicMac^® [13,51,52]. MicMac^® is a FOSS (Cecill-B license) photogrammetric suite developed by IGN^® (French National Geographic Institute) and ENSG^® (French National School for Geographic Sciences). It can be used in a variety of 3D reconstruction scenarios [13]. MicMac^® allows the creation of both 3D models and ortho-imagery. The software, which aims to be a cross-platform project, can run on all main operating systems (Windows, Mac OS, and Linux), although, in our experience, has revealed more stable and complete under Linux environment. MicMac^® processing chain is completely under user control and most of the parameters can be fine-tuned. MicMac^® comprises several tools, each of which is described in the dedicated wiki page [53]. For TP extraction MicMac^® uses the Pastis algorithm that is no more than an interface to the well-known SIFT++, a lightweight distribution of the Scale-invariant feature transform (SIFT) [54]. Based on advances in image feature recognition, characteristic image objects can be automatically detected, described, and matched between images. After that, Apero^® starts from tie points generated by Pastis and computes external and internal orientations compatible with these measurements [55]. The knowledge of the interior orientation of a camera used for image acquisition is a fundamental requisite for precise photogrammetric object reconstruction. Parameters such as principal distance, principal point coordinates regarding the image coordinate system, and some correction terms for lens distortion, etc., are determined by camera calibration [56]. Nowadays, photogrammetric camera calibration is usually carried out along with the calculation of object coordinates within a self-calibrating bundle adjustment. In MicMac^® several distortion models are implemented, including Radial, Fraser, and Fisheye. All theoretical and practical aspects concerning bundle block adjustment with MicMac^® are described in MicMac^® official documentation [13,55]. The main processing steps of MicMac^® can be summarized as follows:

• Tie point computation: the Pastis tool uses the SIFT++ algorithm [57] for the tie point pair generation. This algorithm creates an invariant descriptor that can be used to identify the points of interest matching them even under a variety of perturbing conditions (scale changes, rotation, changes in illumination, viewpoints, or image noise). In this work, this was achieved with Tapioca, a tool interface of SIFT++,
• External orientation: in this step external orientations of the cameras are computed. The relative orientations were computed with the Tapas tool following the free-network approach; this approach involves a calculation of the exterior parameters in an arbitrary coordinate system [58],
• Bundle Block Adjustment: this step includes also the internal parameters, and, for this reason is know as “Self-Calibration”; this is conducted by introducing at least three control points and integrate them within the computation matrix. MicMac^® solves the BBA with the Levenberg–Marquardt (L-M) method [59]. The L-M method is in essence the Gauss–Newton method enriched with a damping factor to handle rank-deficient Jacobian matrices [60]. This stage was achieved by exploiting GCPBascule and Campari tools.

2.2. Agisoft Metashape Photogrammetric Processing

To compare the quality of the generated point cloud, the same dataset was also processed in commercial software, namely Metashape^® software package by Agisoft (ver. 1.7.2 build 12070, 2021) [6]. Metashape^® is a stand-alone software product that performs the photogrammetric processing of digital images and generates 3D spatial data to be used in GIS applications, cultural heritage documentation, and visual effects production as well as for indirect measurements of objects of various scales [6]. The standard photogrammetric pipeline of Metashape is reported below:

• The first step of the photogrammetric processing starts with feature matching across the images: Metashape detects points in the source images which are stable under viewpoint and lighting conditions and generates a descriptor for each point based on its local neighborhood; then, these descriptors are used later to detect correspondences across the photos. This is similar to the well-known SIFT approach but uses different algorithms for a slightly higher alignment quality [61],
• The second stage comprehends the computation of camera intrinsic and extrinsic orientation parameters: Metashape uses a proprietary algorithm to find approximate camera locations and refines them later using a bundle-adjustment algorithm. This should be similar to Bundler algorithm by Snavely et al. (see [62,63]).

The specific algorithms implemented in the software package are not detailed in the manual for business reasons; nevertheless, a description of the Structure-from-Motion (SfM) procedure in Metashape and commonly used parameters are described in [64,65]. It can be noted that during the alignment step, the accuracy was set to high and not to highest; this was to provide the same conditions with respect to MicMac^® processing (TP search on full resolution images); in fact, according to Metashape manual [6], at high accuracy setting the software works with the images at the original size, while highest accuracy setting provides an upscaling of the images by a factor of 4. The software is user-friendly, but the adjustment of parameters is limited to pre-defined values. It can be noted that the user manual describes mainly the general workflow and gives only very limited details regarding the theoretical basis of the underlying calculations and the associated parameters as confirmed in [49,58]. On the contrary, at each step of MicMac^® processing, the user can choose any numerical value, whereas Metashape^® only offers preset values; for example, during the alignment step one can choose among ultra-low, low, medium, high, and ultra-high, whereas MicMac^® allows a multitude of choices. According to [66], the alignment time depends mainly on the number of images while densification step depends also on their resolution and overlap. Those characteristics of the dataset determine the RAM capacity needed, which often is the hardware bottleneck when dealing with photogrammetric commercial software packages. In contrast to commercial solutions, MicMac^® is generally less demanding in RAM given that it writes all temporary files on the physical memory instead of storing them in the cache memory. In the present study, both photogrammetric processes were computed on the same workstation, namely an Asus computer with 16 Gb Ram, 1 Tb SSD storage, Intel Core i5-3330 CPU at 3.00GHz processor, and Nvidia Quadro 600 GPU. Metashape^® does not support the graphic processing unit of the workstation employed in this work [66]; on the other hand, according to [53], MicMac^® provides support for GPU processing starting from dense matching. Hence, to assure the same processing conditions, we do not use GPU support at all.

3. Case Study

3.1. Ponte della Cerra Overpass (Italy)

The object surveyed is an overpass in the Vomero neighborhood of Naples, Italy, as shown in Figure 2; it was built in the seventeenth century and is functional to realize the overpass of Suarez street over Conte della Cerra street. The overpass structure comprises Neapolitan yellow tuff blocks; the geometric model can be assimilated to a barrel vault whose generating curve is a lowered arch with a span light of 16.50 m with a belt deflection of 3.20 m. The whole width, comprising the left and right abutments, is equal to 21.40 m. The overpassing roadway comprises a single carriageway, with one lane in each direction and a total width of 13.45 m, including parking areas, bus stops, and two sidewalks each of which 3.27 m wide. The structure is subject to degradation phenomena and, for this reason, was secured by applying a tessellated mesh to prevent the detachment of the damaged parts. It also has some prestigious coats of arms on both facades. The survey presented here is finalized to the geometric measurements and 3D modeling useful for the structural studies conducted in [67].

3.2. Unmanned Aerial Vehicle Photogrammetric Flight

For the field test, as shown in Figure 3, a consumer UAV from DJI model Mavic 2 Pro was used to survey the test area. It weighs about 1 kg and is equipped with a 1${}^{\prime }$${}^{\prime }$ CMOS Hasselblad optic sensor. The characteristics of the sensor are reported in

Table 2. Camera features and image settings.

Camera Model	Hasselblad L1D-20c
Focal length	10.3 mm
Image format	jpeg
Image width	5472 pixel
Image height	3648 pixel
Exposure time	1/80 s
ISO sensitivity	400
Pixel size	2.41 μm × 2.41 μm

. For exhaustive technical documentation, one can refer to [68]. The UAV is equipped with a dual-frequency multi-constellation (GPS and GLONASS) receiver which in normal scenarios delivers geotagged images with an accuracy of about 10 m; in this particular application, since the location of the survey is a highly degraded urban canyon where the UAV was surrounded by tall buildings, the geotagging procedure failed for several images; moreover, part of the survey was conducted under the overpass removing all chance to fix the GNSS position at all; therefore, after some failures in the alignment step, the coordinates stored in the EXIF file attached to each image was erased. This procedure allows the success of the photogrammetric process but drastically increases computing times, since the software, ignoring the position of each picture taken must search homologous points for every pair of images. In normal conditions, the availability of the position of the takes speeds up the processing since the software searches for homologous points only in overlapped pairs of images.

3.3. Dataset Description

The acquired dataset comprises both horizontal, nadiral, and oblique images. According to [69], to improve detail reconstruction and minimize holes in the 3D model, it is convenient to collect also oblique images. Indeed, oblique images are useful to reduce the shadow areas in which data can not be acquired, especially when dealing with complex objects, as in this case. In particular, the following types of images were acquired, depending on the part of the overpass being reconstructed:

• For the south facade, 111 images (12 oblique and 99 horizontal) of which there are:

  −

  21 images at 10 m from the object;

  −

  90 images at 4 m from the object.

• For the north facade, 70 images (29 oblique and 41 horizontal) of which there are:

  −

  32 images at 10 m from the object;

  −

  38 images at 4 m from the object.

• For the extrados, 41 images at a flying altitude of 40 m of which there are:

  −

  29 nadiral;

  −

  12 oblique.

The photogrammetric survey was conducted according to Comité International de Photogrammétrie (CIPA) 3 × 3 Rules (see [70]). These simple rules, written, tested, and published at the CIPA Symposium in Sofia in 1988, should be observed for photography with non-metric cameras. The rules are divided into geometric rules (control, wide-area stereo photo cover, and detail stereo photo cover), camera rules (camera properties, camera calibration, and image exposure), and procedural rules (record photo layout, log the metadata, and archive). To reach a proper level of detail on the facades presenting some details such as coats-of-arms, images were taken at two different distances, as suggested in [70]; at 10 m of distance the wide-area stereo photo cover while at 4 m close-up images for detail stereo photo cover.

A critical aspect when dealing with the 3D reconstruction of such complex objects where images of facades were taken at very low height as if they were captured like a terrestrial acquisition scheme is to ensure that the algorithm can identify the same key points as tie points between images. To achieve this, the survey strategy focused on having a noteworthy image overlap, both cross- and along-track. Therefore the mean along-overlap was equal to 80% while the cross-track to 55%.

3.4. Ground Control Points

The scaling procedure of the 3D model was created based on a support survey conducted with a professional TOPCON total station. This choice was influenced by two main factors: the impracticability of employing a GNSS receiver due to the highly degraded urban canyon, as well as the type of object we were attempting to model. Thus, an open polygonal was created, consisting of five points on the ground, as shown in Figure 4; then, making station on those points, a detailed survey was carried out to measure several support points, from which 12 of them were selected to model scaling: 4 points on the south facade, 4 points on the north facade, and 4 points on the extrados. The GCPs taken were chosen with a preference for well-identifiable features such as tuff edges and net-holding anchors as shown in Figure 5, Figure 6 and Figure 7. All the points surveyed were taken using a local coordinate system. Attention was paid to be sure that points that were gathered were also visible in the model. Given the historicity of the overpass, we were not allowed to place dedicated targets during the survey; thus, the accuracy related to the natural points chosen was set to 0.005 m, to take into account the uncertainty associated with the type of feature.

4. Results

4.1. Internal and External Orientation Results

In Figure 8a, visual representation of the orientation results of BBA by Metashape^® and MicMac^® is shown. Panel (a) reports the image poses represented with blue rectangles oriented based on Metashape^® external orientation; panel (b) depicts the camera poses, represented with pyramid-shaped 3D objects, computed by MicMac^®. The images reported here have a different visualization given that the two software use different strategies to display camera poses information.

To choose the best camera calibration model for the specific case study, various relative orientation processes were carried out: Radial, Fraser, Four15x4, Brown, and Ebner. Some of them did not converge at all while others exhibit high image residuals. For this specific dataset, it was found that the best trade-off between convergence achievement and residual minimization was the Fraser model [71]. It is a radial model for camera self-calibration, with decentric and affine parameters. In particular, 12 degrees of freedom are taken into account: 1 for focal length (F), 2 for principal point ($P{P}_1$ and $P{P}_2$), 2 for distortion center ($Cdis{t}_1$ and $Cdis{t}_2$), 3 for coefficients of radial distortion ($r_3$, $r_5$, and $r_7$), 2 for decentric parameters ($P_1$ and $P_2$), and 2 for affine (in-plane distortion) parameters ($b_1$ and $b_2$). The parameters computed by the self-calibration algorithm implemented in MicMac^® and then imposed in Metashape^® are reported in

Table 3. Fraser camera model self-calibration parameters.

Parameter	Symbol	Value (pix)
Focal length	F	4276.067
Principal Point coordinates	$P{P}_1$	2702.974
	$P{P}_2$	1836.010
Distortion center coordinates	$Cdis{t}_1$	2686.749
	$Cdis{t}_2$	1809.302
Radial distortion coefficients	$r_3$	$-8.259\times {10}^{-10}$
	$r_5$	$7.756\times {10}^{-17}$
	$r_7$	$4.838\times {10}^{-24}$
Decentric parameters	$P_1$	$2.469\times {10}^{-7}$
	$P_2$	$2.251\times {10}^{-7}$
Affine parameters	$b_1$	$1.017\times {10}^{-4}$
	$b_2$	$1.802\times {10}^{-4}$

.

Parameters computed with the self-calibration method were then imposed in Metashape^® camera calibration to set the same conditions for results comparison. The Apero^® tool of MicMac^® generates external and internal orientations of the camera. The relative orientations were computed with the Tapas tool in two steps: first on a small set of images and then by using the calibration obtained as an initial value for the global orientation of all images. Then, the Campari command is used to compensate for heterogeneous measurements [13]. In Metashape^®, camera alignment by bundle adjustment is achieved by detecting common tie points and match them on images to compute the external camera orientation parameters for each picture. Then, it proceeds to solve for camera internal and external orientation parameters using an algorithm to find approximate camera locations refining them later using the BBA. Figure 9 depicts the probability density estimation of image residuals expressed in pixels. It was obtained by calculating the normalized histogram of image residuals. The height of each bar is the number of samples of images residual falling in that bin divided by the width of the bin and the total number of elements in the sample. In this way, the area of each bar is the relative number of observations. This type of normalization is commonly used to estimate the PDF. Both histograms were calculated considering a bin width of 0.03 pixels. By observing the figure it can be noticed that MicMac^® image residual distribution is long-tailed, whereas Metashape^® is short-tailed. Indeed, the image residuals standard deviation obtained with MicMac^® is higher. In particular, the image residual standard deviation for MicMac^® is equal to 0.277 pixels, whereas for Metashape^®, it is equal to 0.173 pixels. Regarding the mean image residuals, the two software show similar values: MicMac^® reaches the value of 0.770 pixels and Metashape^® attests to 0.735 pixels.

4.2. TP Clouds Results

The point cloud obtained with the commercial software was used as a reference for the quality assessment of the TP cloud generated by MicMac^®. The analysis presented was conducted with the FOSS CloudCompare ver. 2.11.3 (Anoia 64-bits) and Matlab R2021b Statistic toolbox. Various properties of the clouds were compared: number of points, point densities, GCP errors, and CP errors.

Table 4. Summary of the main properties of the TP clouds.

		Number of Points (Points)	Mean Surface Density (Points/m2)	Std Surface Density (Points/m2)
MicMac®	extrados	648,197	7503	8276
	north facade	660,720	8863	7537
	south facade	2,265,025	32,192	25,250
Metashape®	extrados	243,213	1410	1004
	north facade	263,958	1947	984
	south facade	430,178	2428	1489

depicts the main properties of the resulting TP clouds.

Table 4 demonstrates that the MicMac^® TP cloud has significantly more points than the Metashape^® cloud; this is addressed to the way the points are managed in the two software packages. Indeed, Metashape^® merges TPs to obtain a single point via the gradual selection algorithm, whereas MicMac^® does not realize this merging; in fact, if one zooms in a MicMac^® TP and/or dense cloud, a cluster of points will be found. A post-processing decimation can be conducted in MicMac^® via the tool Schnaps to clean and reduce tie points. This reflects in the other metrics reported here (computed taking into account a sphere with a radius of 0.1 m): the surface density of the TP cloud generated by MicMac^® has a mean of 7503 points/m${}^2$, 8863 points/m${}^2$, and 32,192 points/m${}^2$ for the extrados, the north, and the south facades, respectively. The corresponding values for the Metashape^® TP cloud are 1410 points/m${}^2$, 1947 points/m${}^2$, and 2428 points/m${}^2$. Concerning the standard deviations of the surface’s densities, the MicMac^® TP cloud shows values equal to 8276 points/m${}^2$, 7537 points/m${}^2$, and 25,250 points/m${}^2$, for extrados, north, and south facades, respectively. Those values for the Metashape^® TP cloud are 1004 points/m${}^2$, 984 points/m${}^2$, and 1489 points/m${}^2$, respectively. Lastly, another key aspect to take into account is the computation time. According to the Metashape^® manual [66], it does not support the graphic processing unit of the workstation employed in this work; on the other hand, according to [53], MicMac^® provides support for GPU processing starting from dense matching. Hence, to assure the same conditions, we do not use GPU support at all. MicMac^® took 23.48 h to complete the photogrammetric pipeline, while Metashape^® took 45% less time, namely 12.34 h.

In Figure 10 are shown the TP clouds obtained with both software. As confirmed by graphical visualization, the MicMac^® cloud is denser, especially on the facades. Moreover, Metashape^® reveals a more homogeneous reconstruction of the whole model, whereas MicMac^® reveals some holes in intrados reconstruction.

Table 5. Summary of MicMac^® and Metashape^® residuals on GCPs.

Marker Label	MicMac® 3D Err. (m)	Metashape® 3D Err. (m)
P07	0.017	0.026
P09	0.014	0.023
P18	0.018	0.026
P33	0.005	0.013
P35	0.004	0.004
P54	0.005	0.006
P55	0.005	0.009
P60	0.023	0.025
P63	0.006	0.009
P66	0.007	0.012
P67	0.003	0.013
P68	0.013	0.009

and

Table 6. Summary of MicMac^® and Metashape^® residuals on CPs.

Marker Label	MicMac® 3D Err. (m)	Metashape® 3D Err. (m)
P20	0.020	0.023
P50	0.048	0.017
P64	0.024	0.034
P69	0.055	0.052

depict the 3D errors in GCPs and CPs for both software packages. Regarding MicMac^® GCPs errors, except for marker P60, all 3D errors are less than 0.018 m; on the other hand, regarding Metashape^®, markers P07, P09, P18, and P60 exhibit a 3D error of about 0.025 m. Three of these belong to the south facade suggesting that this was the most difficult part to reconstruct. This is confirmed by analyzing the MicMac^® errors on the same markers: except for these and for marker P68, they are the only errors greater than 0.010 m. Table 6 shows that, except for marker P50, 3D errors on CPs are comparable between software packages.

Table 7. Statistics on ground control points and check points errors.

	GCP 3D Error		CP 3D Error
	Mean (m)	Std (m)	Mean (m)	Std (m)
MicMac®	0.010	0.007	0.037	0.017
Metashape®	0.015	0.008	0.031	0.015

contains the corresponding error statistics. The errors are of the same order of magnitude. Regarding the GCPs, MicMac^® shows smaller error statistics; in fact, the mean 3D error and the standard deviation are 0.010 m and 0.007 m, respectively; the same metrics for Metashape^® are 0.015 m and 0.008 m. Conversely, regarding the CPs, the mean 3D error for MicMac^® is 0.037 m with a standard deviation of 0.017 m while the mean 3D error on CPs obtained with Metashape^® is 0.031 m with a standard deviation equal to 0.015 m.

4.3. Relative Accuracy

To assess the relative accuracy of both reconstructed models, four measurements were computed between various GCPs and CPs. The true distances were calculated on the basis of the total station support survey; afterward, the same measurements were realized on the 3D models exploiting CloudCompare software. In Figure 11 is reported an example for the measurement P7-P9 while

Table 8. Relative accuracy measurements summary.

Control Line		MicMac®		Metashape®
	True Dist. (m)	Meas. (m)	3D Err. (m)	Meas. (m)	3D Err. (m)
P7–P9	2.389	2.394	0.005	2.396	0.007
P11–P18	15.396	15.390	−0.006	15.405	0.009
P9–P35	5.442	5.438	−0.004	5.460	0.018
P56–P68	6.959	6.941	−0.018	6.968	0.009

summarizes all the measurements carried out. The errors obtained are comparable; however, it can be noted that Metashape^® shows always positive values, whereas for MicMac^®, three out of four values are negative. Moreover, the errors for Metashape^® ranges from 0.007 m to 0.018 m while for MicMac^® from 0.004 m to 0.018 m.

4.4. Cloud-to-Cloud Distance

In this section, the Cloud-to-Cloud distance, computed exploiting the M3C2 plugin of CloudCompare software package [72], is shown. The principle of nearest neighbour distance is used to compute distances between two points: for each point in the compared cloud, the nearest point in the reference cloud is searched and their Euclidean distance is computed [73]. Figure 12 shows the results of Cloud-to-Cloud distance computation. In the present case, the radius of the sphere where the algorithms search for the nearest neighbors was set to 0.100 m, the reference cloud chosen is the Metashape^® one, and the graphic results are visualized in color scale. The figure provides a visual representation of the Cloud-to-Cloud distance; by analyzing the figure, it emerges that the Cloud-to-Cloud distances are homogeneous mainly on the facades while some misalignment between TP clouds is present under the intrados; nonetheless, this misalignment is under 0.002 m. Figure 13 shows the Cloud-to-Cloud distance probability density function (PDF) estimate between MicMac^® and Metashape^® TP clouds; the mean distance value between the two clouds is equal to 0 mm while the standard deviation is equal to 0.254 mm. Observing the figure, it can be noted that more than 95% of distance values are included in the interval [−1, 1] mm. These results confirm the high-level performance reachable by MicMac^®, given that the two TP clouds are averagely superimposable.

4.5. Other Products

Both software can produce several other products derived from the TP cloud. First, the densification algorithms can generate the dense cloud. After that, textured and tiled models can be obtained. In addition, raster products are obtainable, such as Digital Elevation Model (DEM) and Orthophotos. In this section, for example, the dense clouds generated by both software are shown and their main features are compared. In Figure 14 and Figure 15 are reported the 3D views of the dense clouds obtained with MicMac^® and Metashape^®, respectively.

Table 9. Summary of the main properties of the dense clouds.

		Number of Points (Points)	Mean Surface Density (Points/m2)	Std Surface Density (Points/m2)
MicMac®	extrados	12,445,984	170,382	30,687
	north facade	19,490,308	268,875	175,817
	south facade	51,604,250	1,000,355	480,666
Metashape®	extrados	10,543,654	133,463	101,654
	north facade	9,341,169	118,795	65,411
	south facade	38,479,447	534,579	224,562

shows the number of points and the surface densities after densification. According to the Metashape^® manual [66] and the MicMac wiki [53], to ensure proper comparability, the same parameters were employed: in Metashape^®, the accuracy was set to high, whereas in MicMac^®, when launching the Malt tool, the parameter ZoomF was set to 2. As can be seen in Table 9, for the extrados, the dense cloud obtained with MicMac^® has about 20% more points than the Metashape^® dense cloud. Regarding the north facade the FOSS-derived dense cloud has two times the points than the commercial one. Lastly, the MicMac^® dense cloud of the south facade has about 35% more points than the corresponding from Metashape^®. However, a comparison between dense clouds from different software packages is something hard to achieve for the different algorithms and strategies employed; hence, for this product, we limit to show the resulting values. To deepen this specific aspect a dedicated study should be conducted.

5. Conclusions and Future Works

Unmanned aerial vehicles are playing a major role in the acquisition of geospatial information. With the significantly lower costs associated with UAV surveys and the high quality of the derived products, this is a technique that proves to be useful for several applications and most certainly a valid alternative to traditional techniques. Due to the payload limitation of prosumer UAVs, the reliability of results may not be as high as other techniques, e.g., classical aerial photogrammetry and LiDAR. For this reason, an accuracy assessment of different software packages can reveal truly interesting. The present work represents a case study of the reconstruction of a historical overpass in Italy carried out with MicMac^® open source software and then compared to Metashape^® commercial photogrammetry solution. Particular attention was paid to the image acquisition procedure since, as well-known in literature, it has a relevant impact on the precision of the results.

The quality of the photogrammetric results were assessed by analyzing image residuals, the statistics of GCPs and CPs errors, the relative accuracy assessment, and the Cloud-to-Cloud distance. Regarding the mean image residuals, the two software show similar values: MicMac^® reaches the value of 0.770 pixels and Metashape^® attests to 0.735 pixels. The standard deviations are 0.277 pixels and 0.173 pixels for MicMac^® and Metashape^®, respectively. Analyzing the 3D errors on CPs, MicMac^® revealed slightly worse statistics; in fact, the mean 3D error is equal to 0.037 m with a standard deviation of 0.017 m while the mean 3D error on CPs obtained with Metashape^® is 0.031 m with a standard deviation equal to 0.015 m. For what it concerns the relative accuracy assessment, four linear distances were measured on the 3D models revealing mean errors of 0.008 m and 0.011 m for MicMac^® and Metashape^®, respectively. Lastly, the Cloud-to-Cloud distance analysis was carried out between the TP clouds: it can be noted that more than 95% of distance values are included in the interval [−1, 1] mm. The probability density estimate has a zero mean. Besides this numerical analysis that demonstrates similar results, a visual investigation reveals that the Metashape^® TP cloud presents fewer holes than MicMac^® TP cloud, especially under the overpass. A relevant advantage of MicMac^® resides in the accessibility to intermediate results and in the capability of finely tuning practically all parameters. The trade-off of this extreme manageability lies in the difficulty of usage. Conversely, the nontrivial license cost of the commercial software is justified by its ease of use and its user-friendly interface, which allows anyone with minimal experience to perform the photogrammetric 3D model reconstruction but has few intermediate output capabilities and limited parameters control. It should be highlighted that a comparison between software is something hard to achieve given the secrecy of the commercial software, the long computation times, and the theoretical difference underlying the approaches. Moreover, the object reconstructed is particularly complex being a historical overpass with several details. Lastly, the overpass is placed in an urban canyon where the GNSS support can not be exploited and the scenario changes continuously due to vehicular traffic. The analyses conducted show that the two software packages have comparable products qualities even if the software approaches are different: an open-source project fully customizable versus a commercial black-box software; hence the experiment confirms the potentiality of FOSS for photogrammetric applications. We can state that MicMac^® can produce professional results comparable with products generated by a market-leading commercial solution. The future work will consider other types of objects to reconstruct via photogrammetry. This will define more completely the behaviour of the FOSS investigated when dealing with different geometries. Moreover, in-depth analyses of other products, e.g., DEM and Orthomosaic, will be carried out. Another interesting future goal will be to further improve the cost-effectiveness of the equipment; this can be achieved by employing a compact ultra low-cost UAV. Indeed, in the literature, there are few relevant studies concerning the utilization of a cost-effective platform for surveying and 3D reconstruction, so far. This can represent a promising avenue of research to define a correct procedure for 3D model generation and quality assessment employing ultra-low-cost equipment.