A Robotized Raspberry-Based System for Pothole 3D Reconstruction and Mapping
Abstract
:1. Introduction
- Vision-based methods are fed input visuals (image or video) and include image processing and deep learning algorithms to determine the presence of potholes [18,19,20]. These methods are limited in measuring potholes’ geometric properties, such as depth and volume, because they use two-dimensional (2D) information. In other words, a 2D image or video can only provide us with a two-dimensional representation of a pothole, which makes it difficult to determine its shape and size;
- Vibration-based methods use data collected from inertial sensors inside test vehicles and assume potholes as a significant acceleration impulse [21,22]. However, these methods cannot detect road surfaces (only wheel paths) [23] and the proper identification of abnormal vibration causes (e.g., distinguishing between potholes and pavement bumps) [24];
- Three-dimensional (3D) reconstruction-based methods generate pothole 3D representations, from multiple images of the same scene, in the form of point cloud models, mesh models, and geometric models. By using stereovision-based methods—including photogrammetry and Structure-from-Motion (SfM) techniques—it is feasible to assess the pothole size, shape, and depth and measure its volume accurately. Some commonly used 3D pothole reconstruction-based methods include LiDAR (Light Detection and Ranging) scanning [25], stereo vision [26], and photogrammetry [27]. Various techniques are employed to extract depth and geometric information from images; by identifying corresponding points or features in multiple images, the camersas’ position and orientation is determined, and the 3D structure of the scene can be reconstructed [28]. The quality and accuracy of the 3D model depend on factors such as the number and distribution of images, image resolution, camera calibration, and the effectiveness of feature-matching algorithms [29].
- The photogrammetry approach has been successfully tested by Tion et al. [30] to classify pothole samples in terms of severity levels. In recent years, Atencio et al. [31] developed a method for pothole measurement from 3D models generated from photographs shot by unmanned aerial vehicles (UAVs), determining the optimal flight parameters.
The InfraROB Project
2. The Proposed Integrated System
2.1. Hardware Platform
2.1.1. GPS Module
2.1.2. Camera Module
2.2. Data Acquisition
Pothole Images Acquisition Using Python Script
2.3. Photogrammetric Data Processing
2.4. Computation of Pothole Fill Material
2.5. GIS Technologies to Map Potholes
3. Results
3.1. Laboratory Tests
3.2. On-Site Tests
4. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Bruno, S.; Colonnese, S.; Scarano, G.; Del Serrone, G.; Loprencipe, G. Pavement Distress Estimation via Signal on Graph Processing. Sensors 2022, 22, 9183. [Google Scholar] [CrossRef] [PubMed]
- Basri, E.I.; Razak, I.H.A.; Ab-Samat, H.; Kamaruddin, S. Preventive Maintenance (PM) Planning: A Review. J. Qual. Maint. Eng. 2017, 23, 114–143. [Google Scholar] [CrossRef]
- Fengier, J.; Słowik, M.; Pożarycki, A. Contactless Approach to Determine Pavement Skid Resistance for Pavement Management System. In MATEC Web of Conferences; EDP Sciences: Les Ulis, France, 2018; Volume 222, p. 01003. [Google Scholar]
- Cafiso, S.; Di Graziano, A.; Goulias, D.G.; D’Agostino, C. Distress and Profile Data Analysis for Condition Assessment in Pavement Management Systems. Int. J. Pavement Res. Technol. 2019, 12, 527–536. [Google Scholar] [CrossRef]
- Loprencipe, G.; Cantisani, G.; Di Mascio, P. Global Assessment Method of Road Distresses. In Life-Cycle Design, Assessment, and Maintenance of Structures and Infrastructure Systems, Proceedings of the International Symposium Life-Cycle Civil Engineering, Lake Como, Italy, 11–14 June 2015; Furuta, H., Frangopol, D.M., Akiyama, M., Eds.; CRC Press: Boca Raton, FL, USA, 2015; pp. 1113–1120. [Google Scholar]
- Mouaket, I.M.; Al-Mansour, A.; Sinha, K.C. Evaluation of the Cost-Effectiveness of Pavement Surface Maintenance Activities; Indiana Department of Transportation: West Lafayette, IN, USA, 1990; p. 303. [Google Scholar]
- ASTM D6433-20; Standard Practice for Roads and Parking Lots Pavement Condition Index Surveys. ASTM: West Conshohocken, PA, USA, 2020.
- Biswas, S.; Hashemian, L.; Bayat, A. Investigation of Pothole Severity and Maintenance Methods in Canada through Questionnaire Survey. J. Cold Reg. Eng. 2018, 32, 04018002. [Google Scholar] [CrossRef]
- Kanoungo, A.; Sharma, U.; Goyal, A.; Kanoungo, S.; Singh, S. Assessment of Causes of Pothole Development on Chandigarh Roads. J. Inst. Eng. India Ser. A 2021, 102, 411–419. [Google Scholar] [CrossRef]
- Bosurgi, G.; Modica, M.; Pellegrino, O.; Sollazzo, G. An Automatic Pothole Detection Algorithm Using Pavement 3D Data. Int. J. Pavement Eng. 2022, 1–15. [Google Scholar] [CrossRef]
- Incidenti Stradali: Ancma, Un Decesso Su 10 Dovuto a Buche e Ostacoli in Strada. Available online: https://www.ansa.it/canale_motori/notizie/sicurezza/2019/07/28/incidenti-stradali-ancma-un-decesso-su-10-dovuto-a-buche-e-ostacoli-in-strada_fa845c55-9280-4172-b323-b5105aca96a3.html (accessed on 16 March 2023).
- Biswas, S.; Hashemian, L.; Hasanuzzaman, M.; Bayat, A. A Study on Pothole Repair in Canada through Questionnaire Survey and Laboratory Evaluation of Patching Materials. Can. J. Civ. Eng. 2016, 43, 443–450. [Google Scholar] [CrossRef]
- Pożarycki, A.; Górnaś, P.; Fengier, J. Mechanistic Approach and LCCA for Pavement Rehabilitation in the Absence of Ability to Change Its Vertical Alignment. Arch. Inst. Inż. Lądowej 2015, 63–76. [Google Scholar]
- Hadjidemetriou, G.; Serrano, M.; Vela, P.; Christodoulou, S. Patch Defects in Images Using Support Vector Machines. Civ. Comp Proc. 2015, 108. [Google Scholar]
- Feng, Z.; El Issaoui, A.; Lehtomäki, M.; Ingman, M.; Kaartinen, H.; Kukko, A.; Savela, J.; Hyyppä, H.; Hyyppä, J. Pavement Distress Detection Using Terrestrial Laser Scanning Point Clouds–Accuracy Evaluation and Algorithm Comparison. ISPRS Open J. Photogramm. Remote Sens. 2022, 3, 100010. [Google Scholar] [CrossRef]
- Garbowski, T.; Gajewski, T. Semi-Automatic Inspection Tool of Pavement Condition from Three-Dimensional Profile Scans. Procedia Eng. 2017, 172, 310–318. [Google Scholar] [CrossRef]
- Kim, T.; Ryu, S.-K. Review and Analysis of Pothole Detection Methods. J. Emerg. Trends Comput. Inf. Sci. 2014, 5, 603–608. [Google Scholar]
- Ibragimov, E.; Lee, H.-J.; Lee, J.-J.; Kim, N. Automated Pavement Distress Detection Using Region Based Convolutional Neural Networks. Int. J. Pavement Eng. 2022, 23, 1981–1992. [Google Scholar] [CrossRef]
- Gajjar, K.; Niekerk, T.; van Wilm, T.; Mercorelli, P. Vision-Based Deep Learning Algorithm for Detecting Potholes. J. Phys. Conf. Ser. 2022, 2162, 012019. [Google Scholar] [CrossRef]
- Tedeschi, A.; Benedetto, F. A Real-Time Automatic Pavement Crack and Pothole Recognition System for Mobile Android-Based Devices. Adv. Eng. Inform. 2017, 32, 11–25. [Google Scholar] [CrossRef]
- Mednis, A.; Strazdins, G.; Zviedris, R.; Kanonirs, G.; Selavo, L. Real Time Pothole Detection Using Android Smartphones with Accelerometers; IEEE: Piscataway, NJ, USA, 2011; pp. 1–6. [Google Scholar]
- Du, R.; Qiu, G.; Gao, K.; Hu, L.; Liu, L. Abnormal Road Surface Recognition Based on Smartphone Acceleration Sensor. Sensors 2020, 20, 451. [Google Scholar] [CrossRef] [Green Version]
- Jo, Y.; Ryu, S. Pothole Detection System Using a Black-Box Camera. Sensors 2015, 15, 29316–29331. [Google Scholar] [CrossRef]
- Kyriakou, C.; Christodoulou, S.E.; Dimitriou, L. Smartphone-Based Pothole Detection Utilizing Artificial Neural Networks. J. Infrastruct. Syst. 2019, 25, 04019019. [Google Scholar] [CrossRef]
- Ravi, R.; Habib, A.; Bullock, D. Pothole Mapping and Patching Quantity Estimates Using LiDAR-Based Mobile Mapping Systems. Transp. Res. Rec. 2020, 2674, 124–134. [Google Scholar] [CrossRef]
- Li, Y.; Papachristou, C.; Weyer, D. Road Pothole Detection System Based on Stereo Vision; IEEE: Piscataway, NJ, USA, 2018; pp. 292–297. [Google Scholar]
- Tan, Y.; Li, Y. UAV Photogrammetry-Based 3D Road Distress Detection. ISPRS Int. J. Geo-Inf. 2019, 8, 409. [Google Scholar] [CrossRef] [Green Version]
- Remondino, F.; El-Hakim, S.F.; Gruen, A.; Zhang, L. Turning Images into 3-D Models. IEEE Signal Process. Mag. 2008, 25, 55–65. [Google Scholar] [CrossRef]
- Remondino, F.; Guarnieri, A.; Vettore, A. 3D Modeling of Close-Range Objects: Photogrammetry or Laser Scanning? Beraldin, J.-A., El-Hakim, S.F., Gruen, A., Walton, J.S., Eds.; SPIE Library: San Jose, CA, USA, 2005; p. 216. [Google Scholar]
- Tiong, P.L.Y.; Mustaffar, M.; Hainin, M.R. Road Surface Assessment of Pothole Severity by Close Range Digital Photogrammetry Method. World Appl. Sci. J. 2012, 19, 867–873. [Google Scholar]
- Atencio, E.; Plaza-Muñoz, F.; Muñoz-La Rivera, F.; Lozano-Galant, J.A. Calibration of UAV Flight Parameters for Pavement Pothole Detection Using Orthogonal Arrays. Autom. Constr. 2022, 143, 104545. [Google Scholar] [CrossRef]
- Fendi, K.G.; Adam, S.M.; Kokkas, N.; Smith, M. An Approach to Produce a GIS Database for Road Surface Monitoring. APCBEE Procedia 2014, 9, 235–240. [Google Scholar] [CrossRef]
- Nautiyal, A.; Sharma, S. Condition Based Maintenance Planning of Low Volume Rural Roads Using GIS. J. Clean. Prod. 2021, 312, 127649. [Google Scholar] [CrossRef]
- Obaidat, M.T.; Ghuzlan, K.A.; Bara’W, A.-M. Integration of Geographic Information System (GIS) and PAVER System toward Efficient Pavement Maintenance Management System (PMMS). Jordan J. Civ. Eng. 2018, 12, 449–460. [Google Scholar]
- Kim, Y.-M.; Kim, Y.-G.; Son, S.-Y.; Lim, S.-Y.; Choi, B.-Y.; Choi, D.-H. Review of Recent Automated Pothole-Detection Methods. Appl. Sci. 2022, 12, 5320. [Google Scholar] [CrossRef]
- Brzeziński, K.; Maślakowski, M.; Liszewski, P. Evaluation of the Volume Measurement Optical Method Suitability for Determining the Relative Compaction of Soils. Civ. Eng. J. 2018, 4, 2052. [Google Scholar] [CrossRef] [Green Version]
- de Oliveira, A.K.V.; Bracht, M.K.; Melo, A.P.; Lamberts, R.; Ruther, R. Evaluation of Faults in a Photovoltaic Power Plant Using Orthomosaics Based on Aerial Infrared Thermography. In Proceedings of the 2021 IEEE 48th Photovoltaic Specialists Conference (PVSC), Fort Lauderdale, FL, USA, 20 June 2021; pp. 2604–2610. [Google Scholar]
- Katsamenis, I.; Bimpas, M.; Protopapadakis, E.; Zafeiropoulos, C.; Kalogeras, D.; Doulamis, A.; Doulamis, N.; Martín-Portugués Montoliu, C.; Handanos, Y.; Schmidt, F. Robotic Maintenance of Road Infrastructures: The Heron Project. In Proceedings of the 15th International Conference on Pervasive Technologies Related to Assistive Environments, Corfu, Greece, 29 June 2022–1 July 2022; pp. 628–635. [Google Scholar]
- Cantisani, G.; D’Andrea, A.; Di Mascio, P.; Moretti, L.; Fiore, N.; Petrelli, M.; Polidori, C.; Venturini, L. Materials Study to Implement a 3D Printer System to Repair Road Pavement Potholes. In New Scenarios for Transport Infrastructure and Systems Transition to Inclusivity, Resilience and Sustainability, Proceedings of the AIIT 3rd International Conference, Rome, Italy, 2022, 15–16 September 2022; Giordano Editore: Rome, Italy, 2023; Volume 69, pp. 91–98. [Google Scholar] [CrossRef]
- TinyMobileRobots. Available online: https://tinymobilerobots.com/ (accessed on 18 April 2023).
- Ghael, H.D.; Solanki, L.; Sahu, G. A Review Paper on Raspberry Pi and Its Applications. Int. J. Adv. Eng. Manag. 2020, 2, 4. [Google Scholar]
- Loprencipe, G.; de Almeida Filho, F.G.V.; de Oliveira, R.H.; Bruno, S. Validation of a Low-Cost Pavement Monitoring Inertial-Based System for Urban Road Networks. Sensors 2021, 21, 3127. [Google Scholar] [CrossRef]
- Bruno, S.; Vita, L.; Loprencipe, G. Development of a GIS-Based Methodology for the Management of Stone Pavements Using Low-Cost Sensors. Sensors 2022, 22, 6560. [Google Scholar] [CrossRef]
- Bruno, S.; Loprencipe, G.; Marchetti, V. Proposal for a Low-Cost Monitoring System to Assess the Pavement Deterioration in Urban Roads. Eur. Transp. Eur. 2023, 91, 1–10. [Google Scholar] [CrossRef]
- Ambrož, M.; Hudomalj, U.; Marinšek, A.; Kamnik, R. Raspberry Pi-Based Low-Cost Connected Device for Assessing Road Surface Friction. Electronics 2019, 8, 341. [Google Scholar] [CrossRef] [Green Version]
- Kulkarni, R.B. Dynamic Decision Model for a Pavement Management System. Transp. Res. Rec. 1984, 8, 11–18. [Google Scholar]
- Wu, W.; Qurishee, M.A.; Owino, J.; Fomunung, I.; Onyango, M.; Atolagbe, B. Coupling Deep Learning and UAV for Infrastructure Condition Assessment Automation. In Proceedings of the 2018 IEEE International Smart Cities Conference (ISC2), Kansas, MI, USA, 16 September 2018; pp. 1–7. [Google Scholar]
- Raspberry Pi 4. Available online: https://www.raspberrypi.com/products/raspberry-pi-4-model-b/ (accessed on 8 February 2023).
- U-Blox NEO-6M GPS Module. Available online: https://www.u-blox.com/en/product/neo-6-series (accessed on 8 February 2023).
- GPSD Libray. Available online: https://pypi.org/project/gpsd-py3/ (accessed on 8 April 2023).
- Raspberry Pi Camera Module 3. Available online: https://www.raspberrypi.com/products/camera-module-3/ (accessed on 25 March 2023).
- Picamera Library. Available online: https://github.com/waveform80/picamera (accessed on 5 April 2023).
- Piexif Library. Available online: https://pypi.org/project/piexif/ (accessed on 5 April 2023).
- ContextCapture-Bentley Systems. Available online: https://www.bentley.com/software/contextcapture/ (accessed on 8 April 2023).
- Pixpro Photogrammetry Software. Available online: https://www.pix-pro.com/ (accessed on 17 April 2023).
- Agisoft Metashape. Available online: https://www.agisoft.com/ (accessed on 17 April 2023).
- MicMac. Available online: https://micmac.ensg.eu/index.php/Accueil (accessed on 17 April 2023).
- Meshroom. Available online: https://alicevision.org/#meshroom (accessed on 17 April 2023).
- VisualSFM. Available online: http://ccwu.me/vsfm/index.html (accessed on 17 April 2023).
- QGIS. Available online: https://docs.qgis.org/3.28/it/docs/user_manual/ (accessed on 23 April 2023).
- EUR-Lex Access to European Union Law (2014). Direttiva 2014/24/EU del Parlamento Europeo e del Consiglio, del 26 Febbraio 2014, Sugli Appalti Pubblici e che Abroga La Direttiva 2004/18/CE Testo Rilevante Ai Fini del SEE. Available online: https://eur-lex.europa.eu/legal-content/IT/TXT/?qid=1485620527972&uri=CELEX:32014L0024 (accessed on 17 April 2023).
Maximum Depth [mm] | Average Diameter [mm] | ||
---|---|---|---|
100–200 | 200–450 | 450–750 | |
13–25 | L | L | M |
25–50 | L | M | H |
>50 | M | M | H |
Raspberry Pin | Sensor Pin |
---|---|
3V3 | VCC |
GND | GND |
GPIO 14 | RX |
GPIO 15 | TX |
Parameters | Settings |
---|---|
Final rigid registration | Registration constraints |
Key point density | High |
Pair selection mode | Default |
Component construction mode | OnePass |
Blockwise colour equalization | Enabled |
Splats | Enabled |
Pothole | Pothole Volume [cm3] | Computed Filling Material [kg] |
---|---|---|
I | 392 | 0.83 |
II | 1024 | 2.17 |
III | 925 | 1.96 |
IV | 745 | 1.58 |
V | 1519 | 3.22 |
VI | 429 | 0.91 |
VII | 1925 | 4.08 |
VIII | 1259 | 2.67 |
IX | 2057 | 4.36 |
X | 415 | 0.88 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Bruno, S.; Loprencipe, G.; Di Mascio, P.; Cantisani, G.; Fiore, N.; Polidori, C.; D’Andrea, A.; Moretti, L. A Robotized Raspberry-Based System for Pothole 3D Reconstruction and Mapping. Sensors 2023, 23, 5860. https://doi.org/10.3390/s23135860
Bruno S, Loprencipe G, Di Mascio P, Cantisani G, Fiore N, Polidori C, D’Andrea A, Moretti L. A Robotized Raspberry-Based System for Pothole 3D Reconstruction and Mapping. Sensors. 2023; 23(13):5860. https://doi.org/10.3390/s23135860
Chicago/Turabian StyleBruno, Salvatore, Giuseppe Loprencipe, Paola Di Mascio, Giuseppe Cantisani, Nicola Fiore, Carlo Polidori, Antonio D’Andrea, and Laura Moretti. 2023. "A Robotized Raspberry-Based System for Pothole 3D Reconstruction and Mapping" Sensors 23, no. 13: 5860. https://doi.org/10.3390/s23135860
APA StyleBruno, S., Loprencipe, G., Di Mascio, P., Cantisani, G., Fiore, N., Polidori, C., D’Andrea, A., & Moretti, L. (2023). A Robotized Raspberry-Based System for Pothole 3D Reconstruction and Mapping. Sensors, 23(13), 5860. https://doi.org/10.3390/s23135860