Smartphone LiDAR Technologies for Surveying and Reality Modelling in Urban Scenarios: Evaluation Methods, Performance and Challenges

Abstract

The aim of the research was to evaluate the performance of smartphone depth sensors (Time of Flight Camera(ToF) and Light Detection and Ranging (LiDAR)) from Android (Huawei P30 Pro) and iOS (iPhone 12 Pro and iPAD 2021 Pro) devices in order to build a 3D point cloud. In particular, the smartphones were tested in several case studies involving the scanning of several objects: 10 building material samples, a statue, an interior room environment and the remains of a Doric column in a major archaeological site. The quality of the point clouds was evaluated through visual analysis and using three eigenfeatures: surface variation, planarity and omnivariance. Based on this approach, some issues with the point clouds generated by smartphones were highlighted, such as surface splitting, loss of planarity and inertial navigation system drift problems. In addition, it can finally be deduced that, in the absence of scanning problems, the accuracies achievable from this type of scanning are ~1–3 cm. Therefore, this research intends to describe a method of quantifying anomalies occurring in smartphone scans and, more generally, to verify the quality of the point cloud obtained with these devices.

1. Introduction

In recent years, smartphones equipped with depth sensors were released onto the consumer market. These sensors were advertised as “LiDAR scanners” for iOS devices and “time-of-flight depth cameras” (ToF cameras) for Android. The sensors were originally used to improve the quality of photos (e.g., improved camera focus, bokeh effect, etc.) and to enable augmented reality applications, but they proved to be suitable for scientific purposes [1,2,3].

Historically, Android smartphones integrated depth sensors and augmented reality applications first. The first smartphone equipped with ToF camera and augmented reality features released in the consumer market was the Lenovo Phab 2 Pro in 2016; subsequently, other devices were released such as the ASUS Zenfone AR in 2017, Oppo RX17 Pro, Honor View 20 in 2018, etc. To support augmented reality (AR) on Android smartphones, Google developed the Tango Project. The Tango technology was based on three fundamental parts: depth sensing, mapping motion and area learning. The first part used an RGB-D sensor to estimate the depth of the images; the second part used inertial sensors (gyroscopes and accelerometers); and the third part refined the position using simultaneous localization and mapping (SLAM) technology [1,4,5]. Despite the great potential of the Tango technology, Google stopped supporting the project in March 2018. The reasons were probably the redundancy of the sensors for the common user and the excessive battery consumption [1]. In 2018, Google replaced Project Tango with ARCore, which detects the depth of environments without active sensors; this is the current technology dedicated to augmented reality on most Android devices. Concerning the iOS market, in 2020, Apple released the iPad Pro 2020 and the iPhone 12 Pro. These devices were the first to be equipped with LiDAR scanners and produced interesting research published in the following years. On the basis of these technologies, several experiments were carried out in scanning and modelling indoor and outdoor environments summarised in the following literature review section.

1.1. Literature Review

In this section, improvements in smartphone scanning technology are presented in chronological order through the literature review.

Diakité & Zlatanova [6] investigated the possibilities of the Google Tango Tablet (a device for developers) to scan and model building interiors to support indoor navigation. The 3D models produced did not have enough detail to support advanced indoor navigation, but simple data processing could integrate basic semantic and topological information into the models.

Tomaštík et al. [7] applied Google Tango technology to forest inventory. The authors used the Lenovo Phab 2 Pro to scan three circular tree test areas (radius equal to 12.62 m) differing in age and tree species composition. Point clouds of tree stems were generated from the scans, and tests showed an RMSE of the diameter at breast height (DBH) lower than 0.02 m.

Also for forest inventory purposes, Hyyppä et al. [8] used the Lenovo Phab 2 Pro to measure the diameter of individual tree stems. The authors measured 121 tree stem diameters using traditional methods and compared them with point clouds generated by the Lenovo Phab 2 Pro. The Lenovo device measurements matched traditional measurements with an RMSE of 0.0073 m and a bias mean of 0.003 m.

Mikita et al. [1] applied ARCore technology and a Xiaomi Mi 8 smartphone to survey and model two boulders that were part of rock outcrops in the Trebícsko Nature Park. Almost all the generated models differed from the reference models (generated by Terrestrial LiDAR Scanning-TLS) by values smaller than 6 cu m for volume and 2 sqm for area.

Tsoukalos et al. [9] tested the possibilities of detecting a 12 sqm room with ARCore and EasyAR (a commercial application for augmented reality). At the end of the experiments, the authors were not satisfied with the produced models and proposed modifications and new technologies to improve the 3D models, such as the use of a depth recognition API (recently implemented on a small part of Android devices as DepthAPI) and the use of LiDAR sensors.

Vogt et al. [10] investigated the capabilities of Apple devices to scan small objects; the authors used the LiDAR scanner and TrueDepth technology of the iPad Pro to scan LEGO bricks of different shapes. Comparing the results with an industrial 3D scanner (Artec Space Spider), Vogt et al. showed that in all cases, the industrial scanner provides better results, but the accuracy of the smartphone may be sufficient depending on the applications.

Spreafico et al. [11] presented research concerning the large-scale 3D mapping capabilities of the iPad Pro LiDAR sensor. Focusing on architectural survey applications, Spreafico et al. scanned a scene consisting of an outdoor emergency staircase connected to a historic building. The point cloud captured with the iPad Pro showed a precision of 0.02 m and an accuracy of 0.04 m, which—importantly—is suitable for architectural mapping at a scale of 1:200.

Riquelme et al. [12] used an iPhone 12 Pro to scan a 26 m mechanically excavated cretaceous marlstone and limestone rock face and extracted rock discontinuities from the point cloud. Riquelme et al. identified in their research that the optimal distance for scanning rocks with the iPhone Pro 12 is less than 3 m and, based on their results, highlighted the device’s great potential for detecting rocky slopes.

Luetzenburg et al. [2] investigated the accuracy of the iPad Pro 2020 and iPhone Pro 12 LiDAR scanner in surveying large natural elements for possible applications in the field of geosciences. Luetzenburg et al. surveyed and reconstructed in 3D the Roneklint cliff in Denmark. The cliff was 130 m long and had a mean height of 10 m. The 3D model reconstructed by scanning with iPad Pro 2020 and iPhone Pro 12 LiDAR presented an accuracy of 0.1 m.

Gollob et al. [3] used the iPad Pro LiDAR scanner for forest inventory purposes. The data acquisition time with the iPad Pro was approximately 7.51 min per sample plot (radius equal to 7 m), trees were mapped with a 97.3% detection rate for trees with a DBH less than 10 cm, and the RMSE of the best DBH measurement was 0.0313 m.

Tavani et al. [13] analysed the performance of the iPhone 12 Pro in geoscience-related applications to replace conventional geological instruments. The authors tested the device’s GNSS, IMU, magnetometer, cameras and LiDAR sensor. Regarding the LiDAR sensor, the authors concluded that it was mostly useful for “soft” applications, such as geoheritage documentation and the production of educational materials.

1.2. Aim and Organization of the Paper

A review of the literature reveals that research in the field of smartphone depth sensors has focused on studying one type of device at a time. There is a clear division between research dedicated to Android and Apple sensors and a general lack of comparative research between these two environments. Furthermore, it can be seen that in recent years, research has neglected the study of ToF cameras mounted on Android devices, even though new applications using ToF have been developed.

Based on these considerations, in this manuscript, a suitable method able to investigate the performance offered by depth sensors mounted on Android and Apple devices in various case studies related to urban environments is described.

The paper is composed of five sections. The section “Materials and Methods” contains the smartphones tested (Section 2.1), the research method pipeline (Section 2.2), the case studies discussed (Section 2.2.1) and the analysis method applied to the acquired point clouds (Section 2.2.2). The third section is devoted to the presentation of the results. Section 3.1 shows the results of the tests conducted in the laboratory, while Section 3.2 shows the results of the tests conducted in the field. Discussions and conclusions are discussed at the end of the paper.

2. Materials and Methods

2.1. Mobile Devices and Scanning Apps

In the experimentation, three devices were used: Huawei P30 Pro (Huawei Technologies Co., Ltd., Shenzhen, China), iPhone 12 Pro and iPAD 2021 Pro (Apple Inc., Cupertino, CA, USA). The devices were selected to represent the (best) LiDAR scanning solutions for Android and iOS operating systems, respectively.

Table 1. Main technical features of the 3 mobile devices used in the experimentation.

Device	Huawei P30 Pro	iPhone 12 Pro	iPad 2021 Pro
Image
Chipset	Huawei HiSilicon Kirin 980	Apple A14 Bionic	Apple M1
RAM	8 GB	6 GB	8 GB
Original operative system	Android 9 EMUI 9.1 Pie	iOS 14	iOS 14
Digital Camera	40 Mp + 20 Mp + 8 Mp	12 Mp + 12 Mp + 12 Mp	12MP + 10MP
Aperture Size	F 1.6 + F 2.2 + F 3.4	F 1.6 + F 2.4 + F 2	F 1.8 + F 2.4
Depth sensor	Sony IMX316 (ToF)	Sony IMX590	Sony IMX590
GNSS Constellation	GPS, GLONASS, BeiDou, Galileo, QZSS	GPS, GLONASS, BeiDou, Galileo, QZSS	GPS, GLONASS, BeiDou, Galileo, QZSS
Frequency	L1/L5	L1/L5	L1/L5
Inertial sensors	Accelerometer, gyroscope, magnetometer	Accelerometer, gyroscope, magnetometer	Accelerometer, gyroscope, magnetometer
Weight	191 g	189 g	466 g
Dimensions	158 × 73 × 8 mm	146.7 × 71.5 × 7.4 mm	247.6 × 178.5 × 5.9 mm

shows the technical characteristics of the 3 devices taken into consideration in this paper.

In smartphone depth sensor scanning solutions, a laser beam with a wavelength in near infrared (NIR) of ~8XX-9XX nm is emitted in a 2D array (e.g., 8 × 8 points in iOS) by a vertical-cavity surface-emitting-laser (VCSEL). The pulse time of flight (dToF) is measured by a Single Photon Avalanche Photodiode (SPAD). The combination of VCEL and SPAD has made the implementation of flash-LiDAR solutions in smartphones possible [2].

3D Live Scanner Pro app (Lubos Vonasek Programmierung) was used to perform the scans with the Huawei P30 Pro; this app allows indoor and outdoor scanning available on all Android device equipped with AR (Augmented Reality). Devices equipped with a LiDAR sensor can capture more details during a survey, calculate depth better and generate more accurate 3D scans (e.g., well-defined contours). The maximum resolution of the point cloud generated by 3D Live Scanner Pro is 2 cm. 3D Scanner App™ (Laan Labs) was used in order to perform the scans with the iPhone 12 Pro and iPad 2021 Pro; with this application, 3D models can be generated using LiDAR data and photos: LiDAR scans can be made in low resolution (~1.5 cm) and high resolution (1 cm).

2.2. Research Methodology

The research method used in this work consists of two main phases: (i) 3D survey by smartphone depth sensors and (ii) analysis of the acquired point clouds.

In phase 1, it is necessary to perform the 3D LiDAR survey to obtain the 3D point cloud of the objects taken into consideration. In order to analyse the quality of the 3D point cloud as the type of object investigated varied, four case studies were analysed. The scans were performed with the three mobile devices and the apps previously described in Section 2.1: 3D Live Scanner Pro for Android and 3D Scanner App™ for iOS. With the Huawei P30 Pro, the scans were performed with a resolution of 2 cm, while with the iPhone 12 Pro and iPad 2021 Pro, the scans were performed with resolutions of 1.5 cm and 1 cm.

In phase 2, the point clouds were segmented in order to isolate the scanned objects; subsequently, the point clouds were analysed by mathematical descriptors able to identify the level of quality of the 3D model.

Finally, a visual analysis was used to identify anomalies in point cloud. The software used for data analysis was CloudCompare (DF R&D/TELECOM Pari-sTech ENST-TSI, Paris, France) [14].

An overview of the pipeline of the research methodology is shown in Figure 1; in the following sections, each step will be described in detail.

2.2.1. Phase 1: 3D Survey by Smartphone Depth Sensors

In phase 1, 3D surveys of several objects in both indoor and outdoor environments were carried out. In particular, four case studies were taken into consideration.

The first case study focused on the scanning of some building material samples. The tests were performed to evaluate the scanning performance of depth sensors under controlled laboratory conditions and reduced sensor movement to a minimum. This test was carried out in the geomatics laboratory of the Polytechnic University of Bari, Taranto, Italy (φ: 40°31′36″ N; λ: 17°16′60″ E). We chose to scan several samples of building materials in an urban environment. In this test, we considered albedo as a parameter characterising the material. The measurement of albedo was documented in [15]. We performed the scans by keeping the mobile devices in the nadir direction to the samples and moving them as little as possible (near-static conditions). We scanned the samples from distances of 0.5 m and 1.5 m and with resolutions of 2 cm (Android), 1.5 cm (iOS) and 1 cm (iOS). By combining scan distances (2) and scan resolutions (3), we obtained 6 different point clouds for each sample. With 10 samples available, a data set of 60 point clouds of samples differing in acquisition distance, resolution and material type (albedo) was obtained. In

Table 2. Features of the material samples studied.

Sample Material	Dimensions [m]	Area [sqm]	Albedo
Smooth cement plaster	0.4 × 0.45 × 0.05	0.180	0.514
Raw cement plaster	0.4 × 0.45 × 0.05	0.180	0.524
White lime plaster	0.505 × 0.365 × 0.01	0.184	0.518
Coloured lime plaster	0.525 × 0.47 × 0.01	0.247	0.510
Tetrafluoroethylene (TFE)	0.245 × 0.19 × 0.001	0.046	0.455
Methacrylate (PMMA)	0.305 × 0.12 × 0.005	0.037	0.433
High-density polyethylene (HDPE)	0.23 × 0.22 × 0.002	0.051	0.623
Frosted glass	0.3 × 0.3 × 0.005	0.090	0.517
Steel	0.205 × 0.3 × 0.008	0.061	0.606
Brass	0.105 × 0.303 × 0.009	0.032	0.661

, we report the features of each sample—dimensions, area, material and albedo—while in Figure 2, an overview of the scanned samples is reported.

In the second case study, a statue in public gardens was scanned. The test was performed to investigate how sensor movement could generate aberrations in the scans. The second test was performed in the public gardens of “Villa Peripato” in the city of Taranto, Italy (φ: 40°28′20″ N; λ: 17°14′80″ E). In the Villa Peripato, we scanned a statue representing an animal (Figure 3). The scans were realised by performing concentric circles to the statue at fixed distances of 2 m and 3 m. During the scans, the smartphones were maintained parallel to the statue and worked at resolutions of 2 cm (Android) and 1.5 cm (iOS). From this test, we obtained a dataset consisting of four point clouds that differed in acquisition distance and resolution.

In the third test, we scanned a room inside the geomatics laboratory ([16]—Figure 4) to verify the performance of depth sensors in scanning an object from the inside. The room has a simple parallelepiped shape with dimensions of 4.85 × 4.15 × 2.95 m and contains three cabinets and an air conditioner. To perform the test, the windows were screened with sheets of paper. To perform the scans, we walked around the room, acquiring in order: the walls, the floor and the ceiling. In this test, we carried out three scans with resolutions of 2 cm (Android), 1.5 cm (iOS) and 1 cm (iOS). We created a dataset consisting of three point clouds.

In the fourth case study, the bases of a Doric column were scanned in an archaeological site in order to investigate the performance of the sensors in scanning a complex free-form object. In particular, the objects taken into consideration concern an archaeological site located in the historic centre of Taranto, Italy (φ: 40°28′26″ N; λ: 17°13′59″ E). This site contains two Doric columns and the remains of a third column consisting of a base and 3 column drums (Figure 5a); the columns are dated to the 6th century BC. In the test, we scanned the remains of the third column (Figure 5b). To perform the scans, we made concentric circles around the remains of the column with a variable distance and tried to capture every detail of the object with smartphones. Three scans were taken with resolutions of 2 cm (Android), 1.5 cm (iOS) and 1 cm (iOS). In this way, it was possible to build a dataset consisting of three point clouds.

2.2.2. Phase 2: Point Clouds Analysis

At this phase, the point clouds were segmented in order to isolate the object of experimentation. If segmentation was not carried out, the values of the mathematical descriptors could be altered in the subsequent processing phase. After the segmentation step, the point clouds were analysed by mathematical descriptors. Indeed, the geometric features of a point cloud can be described by applying a principal component analysis (PCA) to the covariance matrix C of the neighbourhood $p_{i_j}$ of a point $\overline{p}$ of a set of points N_p. The covariance matrix C (Equation (1)) can be defined as a three-dimensional tensor containing the geometric information of a set of points N_p in the neighbourhood $p_{i_j}$ [17,18].

$$C={\left[\begin{array}{cc}{p}_{i_1}& -\overline{p}\\ \dots & \dots \\ p_{i_k}& -\overline{p}\end{array}\right]}^T·\left[\begin{array}{cc}{p}_{i_1}& -\overline{p}\\ \dots & \dots \\ p_{i_k}& -\overline{p}\end{array}\right], i_j\in N_p$$

(1)

where $\overline{p}$ is the centroid of a neighbourhood, $p_{i_j}$, of a set of points N_p (Figure 6). Considering an eigenvector problem, it can be written:

$$C·v_l={\lambda }_l·v_l, l\in \left\{1, 2, 3\right\}$$

(2)

The covariance matrix C is symmetrical and positive semidefinite, and using a PCA, it is possible to extract the three eigenvalues ${\lambda }_1, {\lambda }_2, {\lambda }_3$ [19] from the matrix. The eigenvalues locally describe the 3D structure of the point set N_p and quantify its variation along the direction of the corresponding eigenvector $v_1, v_2, v_3$ (Figure 6).

Through mathematical operations between the three eigenvalues, it is possible to calculate certain form-specific mathematical descriptors called covariance-features or eigenfeatures that are capable of describing certain geometric characteristics of the point cloud. For example, some descriptors found in the literature are linearity, planarity, anisotropy, omnivariance, eigentropy and surface variation and sphericity [17,18,20]. In this research, we used the eigenfeatures of planarity, pmnivariance and surface variation to analyse point clouds based on their shape (eigenfeature analysis). The equations of the three latter descriptors are:

$$\mathrm{Planarity} P_{\lambda }=\frac{{\lambda }_2-{\lambda }_3}{{\lambda }_1}$$

(3)

$$\mathrm{Omnivariance} O_{\lambda }={\left({\lambda }_1·{\lambda }_2·{\lambda }_3\right)}^{\frac{1}{3}}$$

(4)

$$\mathrm{Surface} \mathrm{Variation} S{V}_{\lambda }=\frac{{\lambda }_3}{{\lambda }_1+{\lambda }_2+{\lambda }_3}$$

(5)

Planarity quantitatively describes the tendency of the point cloud to arrange itself along plane surfaces. This eigenfeature can be used to describe point clouds of planar objects or those formed by a combination of planes. For this reason, during data analysis, we used Planarity as a descriptor for the 10 samples in the first test and the room in the third test.

Omnivariance quantifies the degree of inhomogeneity of the point cloud in 3 dimensions. This descriptor can be used to describe the point clouds of freeform objects, and in this work, we applied omnivariance to the statue in the second test and to the remains of the column in the fourth test.

Surface variation quantitatively describes the variations along the normal to the surface of a point cloud and, as demonstrated in Pauly et al. [18], can be used to identify two point clouds side-by-side and overlapping at a certain distance (Figure 7a). Figure 7b shows the descriptor applied to two flat point clouds that are side-by-side with their corners overlapping (the orange ellipse indicates the green overlapping parts identified with surface variation). For these reasons, we used surface variation in all four case studies to identify and try to quantify the phenomenon of smartphone point clouds surface splitting.

Finally, a visual analysis of the point clouds was performed to confirm and visually highlight anomalies and problems in the scans. The analysis was performed by slicing the point clouds in some critical parts indicated by the scalar field generated by the previous eigenfeature analysis. This made the visual analysis easier and more effective.

3. Results

In this section, we report the results obtained in the four case studies. The results were divided into two sections: the first section is dedicated to the tests performed in the laboratory; the second section is dedicated to the field tests and in real conditions. Since the LiDAR depth sensor of the iOS devices (iPhone 12 Pro and iPad 2021 Pro) is the same (Table 1), in this section, we will only discuss the most significant point clouds of the two devices.

3.1. Laboratory Testing under Controlled Conditions

In the first case study, 10 samples of building material were scanned. We reported some examples of the point clouds acquired with 1 cm resolution at a distance of 0.5 m in Appendix A (Figure A1).

Operating on planar samples, it was possible to observe the structure of the point cloud generated by smartphone apps. As can be seen in Figure 8, when iOS scanned with 1 cm resolution (Figure 8a), it generated an ordered point cloud; when working with ~1.5 cm resolution, the device generated a disordered cloud (Figure 8b). Android’s point cloud, shown in Figure 8c, was as ordered as the iOS one in Figure 8a, but of course had a different resolution.

We calculated the mean values of surface variation and planarity for the 60 point clouds obtained from the sample scans. The tables containing the surface variation and planarity values can be found in Appendix A (

Table A1. Mean surface variation values of the scans of the 10 samples.

Sample Material	Scan Distance: 0.5 m			Scan Distance: 1.5 m
	SVλ 1 cm	SVλ 1.5 cm	SVλ 2 cm	SVλ 1 cm	SVλ 1.5 cm	SVλ 2 cm
Smooth cement plaster	0.0020	0.0041	0.0082	0.0016	0.0041	0.0060
Raw cement plaster	0.0024	0.0037	0.0056	0.0016	0.0019	0.0036
White lime plaster	0.0021	0.0022	0.0037	0.0005	0.0013	0.0017
Coloured lime plaster	0.0009	0.0026	0.0021	0.0004	0.0010	0.0008
Tetrafluoroethylene (TFE)	0.0013	0.0034	0.0004	0.0001	0.0001	0.0003
Methacrylate (PMMA)	0.0012	0.0010	0.0003	0.0002	0.0002	0.0003
High-density polyethylene (HDPE)	0.0003	0.0001	0.0001	0.0001	0.0001	0.0002
Frosted glass	0.0004	0.0001	0.0001	0.0002	0.0001	0.0001
Steel	No data	No data	0.1672	0.0005	0.0003	0.0051
Brass	0.0021	0.1455	0.1000	0.0003	0.0007	0.0142

and

Table A2. Mean Planarity values of the scans of the 10 samples.

Sample Material	Scan Distance: 0.5 m			Scan Distance: 1.5 m
	Pλ 1 cm	Pλ 1.5 cm	Pλ 2 cm	Pλ 1 cm	Pλ 1.5 cm	Pλ 2 cm
Smooth cement plaster	0.70	0.53	0.73	0.66	0.52	0.67
Raw cement plaster	0.70	0.54	0.71	0.65	0.53	0.68
White lime plaster	0.69	0.53	0.74	0.67	0.52	0.78
Coloured lime plaster	0.71	0.53	0.80	0.67	0.52	0.84
Tetrafluoroethylene (TFE)	0.67	0.50	0.72	0.65	0.52	0.70
Methacrylate (PMMA)	0.66	0.51	0.63	0.65	0.48	0.73
High-density polyethylene (HDPE)	0.67	0.51	0.76	0.66	0.51	0.75
Frosted glass	0.70	0.54	0.82	0.67	0.52	0.79
Steel	No data	No data	-	0.67	0.52	0.66
Brass	0.63	-	-	0.63	0.53	-

), while below, we propose the data in graphic form for easier exploration and evaluation.

In Figure 9, we reported the calculated surface variation values for almost all samples scanned at a distance of 0.5 m. Special cases of steel and brass will be discussed separately in Section 4. SV_λ values obtained from the materials shown in Figure 9 can be considered excellent, as they are inferior to 0.33, the theoretical maximum value of surface variation [18]. The surface variation values of the samples scanned at 1.5 m were reported in Appendix A (Figure A2). The plaster, plastic and frosted glass data confirmed the excellent values obtained for the 0.5 m scans.

In Figure 10, we reported the planarity values of the samples scanned at 0.5 m distance, while the values of the samples scanned at 1.5 m distance were reported in Appendix A (Figure A3). From the planarity analysis, it can be seen that they take on a value close to unity and thus the tendency of the point cloud to dispose along planar surfaces. Finally, we investigated the existence of a possible linear correlation between the albedo of the materials and their surface variation values.

In

Table 3. R² values obtained by linear regression between sample albedo and surface variation values.

Resolution	R2 for Scanning Distance: 0.5 m	R2 for Scanning Distance: 1.5 m
1 cm (iOS)	0.54	0.68
1.5 cm (iOS)	0.53	0.40
2 cm (Android)	0.27	0.20

, we reported the R² values obtained. In statistics, the value of R² (called coefficient of determination) is a coefficient that indicates how much the variation of a dependent variable may depend on the variation of an independent variable in a regression model. R² values between 0.7 and 0.5 indicate a moderate relationship [21,22], so we concluded that there was a moderate-to-weak relationship between albedo and surface variation for the LiDAR sensor of the iOS devices. In contrast, the relationship for the Android device appeared weaker.

3.2. Field Tests and Applications under Real Conditions

In this section, we reported the data analysis of field tests and performed under real and dynamic conditions (second, third and fourth case study). Examples of the scans of the statue, the room and the rests of the Doric column are presented in Appendix B (Figure A4).

In the analysis phase of the second case study, we focused on the shaft of the statue; therefore, the point cloud was appropriately sectioned in the middle of the shaft for analysis. From the analysis of the point cloud, it was easy to detect and isolate a drift problem for the iOS device during the scanning phase at a distance of 3 m. In Figure 11, the extent of the drift can be estimated using the metric bar. The problem did not occur for the iOS and Android scans at 2 m; the sections of these latter scans are shown in Appendix B (Figure A5a,b). In

Table 4. Surface variation and omnivariance values of the statue, second case study.

Object Scanned	Scan Distance: 2 m		Scan Distance: 3 m
Statue	SVλ 1.5 cm	SVλ 2 cm	SVλ 1.5 cm	SVλ 2 cm
	0.0071	0.0103	0.0417	No data
	Oλ 1.5 cm	Oλ 2 cm	Oλ 1.5 cm	Oλ 2 cm
	0.0012	0.0013	0.0017	No data

, we reported the values of surface variation and omnivariance obtained from the scans; the highest value of SV_λ was associated with the scan where the drift problem occurred. Additionally, observing the omnivariance value, it can be seen that the inhomogeneity of the point cloud increases as the SV_λ value increases.

The scan performed with the Android device at 3 m produced no results (it did not acquire any points); this indicates that the range of the device is less than 3 m. Figure 11b,c presents additional images of the point cloud affected by drift problems. Figure 11d shows a section (C-C) executed in the area of the scan closure, and Figure 11c shows a top view of the statue.

Table 5. Surface variation and planarity values of the point cloud of the lab. room, third case study.

Object Scanned	Scan Distance: Adaptative
Laboratory room	SVλ 1 cm	SVλ 1.5 cm	SVλ 2 cm
	0.0119	0.0109	0.0234
	Pλ 1 cm	Pλ 1.5 cm	Pλ 2 cm
	0.8607	0.8940	0.7849

shows the surface variation and planarity values for the third case study. Figure 12 shows the room planimetry obtained by dissecting the point clouds at a height of approximately 1.50 m (the scalar field represents the planarity values). Looking at the planarity values in the table, it can be seen (see Figure 12) that the Android device performed unsatisfactorily for geomatics purposes. The iOS values, on the other hand, were quite encouraging for the scans performed at both 1 and 1.5 cm.

The fourth case study involved the scanning of the remains of a Doric column located in Taranto. We performed three scans with resolutions of 1 cm (iOS), 1.5 cm (iOS) and 2 cm (Android). In

Table 6. Surface variation and omnivariance values of the point cloud of the rests of the column, fourth case study.

Object Scanned	Scan Distance: Adaptative
Doric column rests	SVλ 1 cm	SVλ 1.5 cm	SVλ 2 cm
	0.0400	0.0682	0.0309
	Oλ 1 cm	Oλ 1.5 cm	Oλ 2 cm
	0.0019	0.0022	0.0018

, we reported the surface variation and omnivariance values obtained for the three scans. Observing the values, it can be seen that the scan made at a resolution of 1.5 cm (iOS) was the worst.

In order to identify possible drift and splitting of the surface, we carried out sections of the point cloud in the vertical (sect. A-A) and horizontal (on the third column drum sect. B-B) scan closing zones (Figure 13).

Figure 14 presents the A-A and B-B sections carried out on the three point clouds. In particular, Figure 14a,d shows the sections taken on the 1 cm resolution scan, Figure 14b,e shows the sections carried out on the 1.5 cm resolution scan and Figure 14c,f shows the sections carried out on the 2 cm resolution scan.

Observing the sections, it can be seen that the scan with a resolution of 1.5 cm (Figure 14b,e) was most affected by surface splitting related to drift problems. Surface splitting occurred in the horizontal direction when the operator scanned the side surface of the object and in the vertical direction when the operator scanned the column drum at the top of the column to close the scan. The 1 cm resolution scan (Figure 14a,d) showed minor surface splitting on the third column drum compared to the 1.5 cm resolution scan. The 2 cm resolution scan appears to be the best of the three; the scan only presented small problems at the junctions between one column drum and another.

In addition, in order to assess the accuracy and precision of the point clouds obtained in the third and fourth case studies, the cloud-to-cloud (C2C) distance was calculated with respect to a point cloud surveyed with the TLS and photogrammetric method.

In the third case study, the 3D point cloud was obtained using a HDS3000 Terrestrial Laser Scanner, which has a position accuracy of 6mm@50m.

In the fourth case study, a photogrammetric survey using digital single lens reflex camera and a structure from motion–multi-view stereo (SfM-MVS) approach was performed [23]. In particular, a Nikon D3300 with a Nikkor 20 mm f/2.8D fixed focal lens was used for the 3D survey [24]. The point cloud was built in an Agisoft Metashape environment; the root mean square equivalent (RMSE), evaluated on six ground control points (GCPs), was 0.001 m.

To compare the point clouds, it was necessary to subsample in order to obtain a statistically fair comparison.

Table 7. Mean and standard deviation values of C2C distance obtained in the third and fourth case studies.

Object Scanned	Resolution: 1 cm		Resolution: 1.5 cm		Resolution: 2 cm
	µ C2C [m]	σ C2C [m]	µ C2C [m]	σ C2C [m]	µ C2C [m]	σ C2C [m]
Laboratory room	0.0334	0.0264	0.0224	0.0449	0.0518	0.0580
Doric column rests	0.0153	0.0132	0.0383	0.0238	0.0127	0.0107

shows the C2C values of mean and standard deviation of C2C distance for the third and fourth case studies between smartphone point clouds and reference point clouds; in addition, in Appendix B, we reported the histograms (Figure A6).

4. Discussion

Scans performed under controlled laboratory conditions on materials in the group of plasters, plastics and frosted glass reported generally excellent SF_λ and P_λ values. This is a positive aspect considering that LiDAR smartphones, in their working conditions, may have to scan different material surfaces in the same scenario (e.g., a building, a square, a street, etc.). Observing the SF_λ values in detail, there is a division between the “plasters” group and the “plastics and frosted glass” group, which have lower values than “plasters”. The difference is very small, and we estimate that this is not a problem for multimaterial scanning.

The steel and brass samples scanned at a distance of 0.5 m and resolutions of 1 cm and 1.5 cm showed considerable noise to the extent that they could not be analysed. Scanning the same samples at a resolution of 2 cm showed a surface variation value of 0.1672 and a cracked and deformed surface (Figure 15a).

Steel and brass samples scanned at a distance of 1.5 m showed generally better surface variation values than scans performed at a closer distance (0.5 m). The 1 cm resolution scan of brass was good, while the 1.5 cm and 2 cm resolution scans produced surface variation values of 0.1455 and 0.1000 due to some splitting and deformation (Figure 15b,c). Problems in scanning metallic materials could be caused by the surface structure; metallic materials can be scanned more easily by increasing the acquisition distance and angle of the laser beam [15].

The planarity values for the Android device are interesting, as this device achieved equal or better results than iOS devices with a lower resolution. The possible deformations of planar surfaces can be better evaluated only in tests performed under dynamic conditions.

Analyses regarding a linear correlation between albedo and surface variation showed a moderate to very weak relationship. This is another positive element since it indicates that the nature of the material does not primarily affect the quality of the scans.

Through tests performed under dynamic conditions, we were able to highlight the problems associated with moving scans. Two main problems emerged: the splitting of surfaces and the incorrect closure of scans caused by inertial navigation system (INS) drift problems. Drift problems depend on calculation of the smartphone’s position. During the scanning process, the position of the smartphone is determined instant by instant based on the previous position estimated by inertial sensors. In this process, small errors of the INS can propagate and cause large positioning errors in the scanning of large objects or large scenarios [13,25]. However, the drift problems in practice also resulted in the surfaces splitting and overlapping in the scan closure areas. Therefore, it can also be said that the two problems are, to a certain degree, related.

Observing the second case study, it can be deduced that increasing the distance to the object being scanned increases the possibility of encountering drift problems. This is verifiable in the case of scans performed on an object along a circular path; furthermore, varying the distance to the object being scanned (e.g., due to obstacles) increases the possibility of the occurrence of a drift effect. The phenomenon just described can also be observed in the scans of two structures collected for documentation purposes. Both structures were scanned with the iPAD Pro 2021 with a resolution of 1.5 cm, as shown in Figure 16.

In the first structure (Figure 16a), due to tight spaces, the operator scanned by standing still in the centre of the structure and rotating in place. In the second structure (Figure 16b), the operator moved actively inside the structure to collect every detail. The scans of the structures are generally good, but when analysing the sections, it is evident that the section of the first structure (Figure 16c) has fewer problems (or no problems) with surface splitting compared to the second (Figure 16d), which has surface splitting at the entrance to the structure, where the scan closed. This is an example of how movement can generate drift problems that practically result in surface splitting.

Finally, through the third and fourth case studies, it can be seen that in general, the accuracies are on the order of 1–3 cm, but scanning problems such as surface splitting can degrade these values to more than double their original values (see Table 7). For these reasons, it is very important to monitor these issues in scans performed with smartphones and the analysis methods described in this paper.

5. Conclusions

In this research, we evaluated and compared the performance of depth sensors mounted on iOS (iPhone 12 Pro and iPad 2021 Pro) and Android (Huawei P30 Pro) devices based on five case studies. In addition, we related mathematical descriptors to specific smartphone point cloud issues in order to quantify anomalies occurring in scans to better understand the point cloud and to simplify visual analysis using scalar fields. This method also allows the evaluation and identification of issues that may affect mobile device scans without additional reference scans that are not available in real applications. In the cases studied, we applied the eigenfeatures of surface variation, planarity and omnivariation; in the case of long and thin structures (e.g., pipes, traffic signals, streetlights etc.), the eigenfeature of linearity should be used.

Generally, mobile devices have proven to be useful tools for scanning objects and environments in urban scenarios; their main benefits are: light weight, manageability, the possibility of having a scanning tool with you at all times, the possibility of quickly sharing the models created with other users, the speed of scanning and the possibility of checking scans directly on site and rescanning them if necessary. The problems encountered can be attributed to the occurrence of certain anomalies, such as surface splitting, loss of flatness and drift problems in the INS. Another problem is the low diffusion of these sensors on smartphones. Currently, only a few devices are equipped with depth sensors. This is related to market demands, so there may be an evolution of these in the coming years.

Observing the third and fourth case studies, we can deduce that the precisions achievable by these sensors are around 1–3 cm in the absence of anomalies. The possible fields of application concern the scanning of small- and medium-sized objects and scenarios, compatible with the movement of a human being, for subsequent 3D modelling.

In this paper, particular insight was applied to some freeform structures; further applications of 3D modelling can be found in Wang et al. [26]. In addition, large objects and scenarios were not considered; therefore, subsequent research will focus on the acquisition of large structures and new apps capable of handling complex scenarios.