Graph Matching for Underwater Simultaneous Localization and Mapping Using Multibeam Sonar Imaging
Abstract
:1. Introduction
- We introduce a novel registration algorithm to effectively address the issue of local optima in inter-frame registration due to the non-convex nature of the Iterative Closest Point (ICP) algorithm.
- We employed an enhanced Accelerated-KAZE (AKAZE) algorithm to improve feature-matching accuracy in sparse environments and along extended straight lines, thereby enhancing the robustness of SLAM.
- We integrated an Inertial Measurement Unit and Doppler velocimeter data for the global state estimation of the robot, incorporating various pose design factors into a factor graph for back-end optimization.
2. Related Work
2.1. Point Cloud Registration
2.2. Underwater Sonar SLAM
3. Inter-Frame Registration
3.1. Graph Matching ICP
3.1.1. Sonar Image Processing and Feature Extraction
3.1.2. Gaussian Descriptors and Matching
Algorithm 1 Matching constraint algorithm |
Require: Descriptor set for two pictures and and abnormal distance threshold 1: , 2: 3: for every in /*i */ 4: 5: 6: 7: for every in /*j */ 8: /*Calculate the similarity between two descriptors*/ 9: 10: if then 11: /*Calculate the similarity between two descriptors*/ 12: 13: if then 14: 15: 16: 17: end if 18: else if then 19: if then 20: 21: 22: end if 23: end if 24: end for 25: if then 26: 27: end if 28: end for 29: 30: return |
Algorithm 2 Similarity scoring algorithm |
Require: Two descriptors and , angle threshold , score distance threshold 1: 2: 3: /*m , n */ 4: while and do 5: /*calculate score*/ 6: 7: 8: if then 9: if then 10: 11: end if 12: , 13: else if then 14: 15: else 16: 17: end if 18: end while 19: return score |
3.2. Parallel Inter-Frame Registration Algorithm Using AKAZE and GM Methods
4. SLAM System
4.1. Sonar’s Field of View
4.2. Underwater Robot Motion Estimation
4.3. Factor Graph and Mapping
5. Results
5.1. Underwater Platform and Datasets
5.2. Inter-Frame Registration Results
5.2.1. GM Results
5.2.2. AKAZE Results
5.3. SLAM Results
5.3.1. Indoor Environment Test Results
5.3.2. Outdoor Environment Test Results
6. Discussion and Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Data Availability Statement
Conflicts of Interest
References
- Wang, Y.; Ji, Y.; Woo, H.; Tamura, Y.; Tsuchiya, H.; Yamashita, A.; Asama, H. Acoustic camera-based pose graph slam for dense 3-d mapping in underwater environments. IEEE J. Ocean. Eng. 2020, 46, 829–847. [Google Scholar] [CrossRef]
- Salvi, J.; Petillo, Y.; Thomas, S.; Aulinas, J. Visual slam for underwater vehicles using video velocity log and natural landmarks. In Proceedings of the OCEANS 2008, Quebec City, QC, Canada, 15–18 September 2008; pp. 1–6. [Google Scholar]
- Beall, C.; Dellaert, F.; Mahon, I.; Williams, S.B. Bundle adjustment in large-scale 3d reconstructions based on underwater robotic surveys. In Proceedings of the OCEANS 2011 IEEE, Santander, Spainm, 6–9 June 2011; pp. 1–6. [Google Scholar]
- Shkurti, F.; Rekleitis, I.; Scaccia, M.; Dudek, G. State estimation of an underwater robot using visual and inertial information. In Proceedings of the 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, San Francisco, CA, USA, 25–30 September 2011; pp. 5054–5060. [Google Scholar]
- Corke, P.; Detweiler, C.; Dunbabin, M.; Hamilton, M.; Rus, D.; Vasilescu, I. Experiments with underwater robot localization and tracking. In Proceedings of the 2007 IEEE International Conference on Robotics and Automation, Roma, Italy, 10–14 April 2007; pp. 4556–4561. [Google Scholar]
- Folkesson, J.; Leonard, J.; Leederkerken, J.; Williams, R. Feature tracking for underwater navigation using sonar. In Proceedings of the 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, San Diego, CA, USA, 29 October–2 November 2007; pp. 3678–3684. [Google Scholar]
- Fallon, M.F.; Folkesson, J.; McClelland, H.; Leonard, J.J. Relocating underwater features autonomously using sonar-based SLAM. IEEE J. Ocean. Eng. 2013, 38, 500–513. [Google Scholar] [CrossRef]
- Richmond, K.; Flesher, C.; Lindzey, L.; Tanner, N.; Stone, W.C. SUNFISH®: A human-portable exploration AUV for complex 3D environments. In Proceedings of the OCEANS 2018 MTS/IEEE, Charleston, CA, USA, 22–25 October 2018; pp. 1–9. [Google Scholar]
- Segal, A.; Haehnel, D.; Thrun, S. Generalized-icp. In Proceedings of the Robotics: Science and Systems, Seattle, WA, USA, 28 June–1 July 2009; Volume 2, p. 435. [Google Scholar]
- Besl, P.; McKay, N. A Method for Registration of 3-D Shapes. IEEE Trans. Pattern Anal. Mach. Intell. 1992, 14, 239–256. [Google Scholar] [CrossRef]
- Chen, Y.; Medioni, G. Object Modeling by Registration of Multiple Range Images. In Proceedings of the 1992 IEEE International Conference on Robotics and Automation, Nice, France, 12–14 May 1992; pp. 2724–2729. [Google Scholar]
- Li, H.; Hartley, R. The 3D-3D registration problem revisited. In Proceedings of the International Conference on Computer Vision, Rio De Janeiro, Brazil, 14–21 October 2007; pp. 1–8. [Google Scholar]
- Bazin, J.C.; Seo, Y.; Pollefeys, M. Globally optimal consensus set maximization through rotation search. In Proceedings of the Asian Conference on Computer Vision, Daejeon, Korea, 5–9 November 2012; pp. 539–551. [Google Scholar]
- Biber, P.; Strasser, W. The normal distributions transform: A new approach to laser scan matching. In Proceedings of the 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No. 03CH37453), Las Vegas, NA, USA, 27 October–1 November 2003; Volume 3, pp. 2743–2748. [Google Scholar]
- Magnusson, M. The Three-Dimensional Normal-Distributions Transform, an Efficient Representation for Registration, Surface Analysis, and Loop Detection. Ph.D. Thesis, Örebro University, Örebro, Sweden, 2009. [Google Scholar]
- Hornung, A.; Wurm, K.; Bennewitz, M.; Stachniss, C.; Burgard, W. OctoMap: An efficient probabilistic 3D mapping framework based on Octrees. Auton. Robot. 2013, 34, 189–206. [Google Scholar] [CrossRef]
- Bouraine, S.; Bougouffa, A.; Azouaoui, O. Particle swarm optimization for solving a scan matching problem based on the normal distributions transform. Evol. Intell. 2021, 15, 1–12. [Google Scholar] [CrossRef]
- Santos, M.M.; Zaffari, G.B.; Ribeiro, P.O.; Drews-Jr, P.L.; Botelho, S.S. Underwater place recognition using forward-looking sonar images: A topological approach. J. Field Robot. 2019, 36, 355–369. [Google Scholar] [CrossRef]
- Eustice, R.; Walter, M.; Leonard, J. Sparse extended information filters: Insights into sparsification. In Proceedings of the 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, Edmonton, AB, Canada, 2–6 August 2005; pp. 3281–3288. [Google Scholar]
- Johannsson, H.; Kaess, M.; Englot, B.; Hover, F.; Leonard, J. Imaging sonar-aided navigation for autonomous underwater harbor surveillance. In Proceedings of the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei, Taiwan, 18–22 October 2010; pp. 4396–4403. [Google Scholar]
- Shin, Y.S.; Lee, Y.; Choi, H.T.; Kim, A. Bundle adjustment from sonar images and SLAM application for seafloor mapping. In Proceedings of the OCEANS 2015-MTS/IEEE, Washington, DC, USA, 19–22 October 2015; pp. 1–6. [Google Scholar]
- Jiang, M.; Song, S.; Li, Y.; Jin, W.; Liu, J.; Feng, X. A survey of underwater acoustic SLAM system. In Proceedings of the Intelligent Robotics and Applications: 12th International Conference, ICIRA 2019, Shenyang, China, 8–11 August 2019; Part II 12. Springer: Berlin/Heidelberg, Germany, 2019; pp. 159–170. [Google Scholar]
- Li, J.; Kaess, M.; Eustice, R.M.; Johnson-Roberson, M. Pose-graph SLAM using forward-looking sonar. IEEE Robot. Autom. Lett. 2018, 3, 2330–2337. [Google Scholar] [CrossRef]
- Joshi, B.; Xanthidis, M.; Roznere, M.; Burgdorfer, N.J.; Mordohai, P.; Li, A.Q.; Rekleitis, I. Underwater exploration and mapping. In Proceedings of the 2022 IEEE/OES Autonomous Underwater Vehicles Symposium (AUV), Singapore, 19–21 September 2022; pp. 1–7. [Google Scholar]
- Alcantarilla, P.F.; Solutions, T. Fast explicit diffusion for accelerated features in nonlinear scale spaces. IEEE Trans. Patt. Anal. Mach. Intell 2011, 34, 1281–1298. [Google Scholar]
- Yang, J.; Li, H.; Campbell, D.; Jia, Y. Go-ICP: A globally optimal solution to 3D ICP point-set registration. IEEE Trans. Pattern Anal. Mach. Intell. 2015, 38, 2241–2254. [Google Scholar] [CrossRef]
- Nitzberg, R. Constant-false-alarm-rate signal processors for several types of interference. IEEE Trans. Aerosp. Electron. Syst. 1972, AES-8, 27–34. [Google Scholar] [CrossRef]
- Ester, M.; Kriegel, H.P.; Sander, J.; Xu, X. Density-based spatial clustering of applications with noise. In Proceedings of the International Conference on Knowledge Discovery and Data Mining (KDD-96), Portland, OR, USA, 2–4 August 1996; Volume 240. [Google Scholar]
- Leordeanu, M.; Hebert, M.; Sukthankar, R. An integer projected fixed point method for Graph Matching and map inference. Adv. Neural Inf. Process. Syst. 2009, 22, 1114–1122. [Google Scholar]
- Johnston, H. Cliques of a graph-variations on the Bron-Kerbosch algorithm. Int. J. Comput. Inf. Sci. 1976, 5, 209–238. [Google Scholar] [CrossRef]
- Zass, R.; Shashua, A. Probabilistic graph and hyperGraph Matching. In Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA, 23–28 June 2008; pp. 1–8. [Google Scholar]
- Zhou, F.; De la Torre, F. Factorized Graph Matching. IEEE Trans. Pattern Anal. Mach. Intell. 2015, 38, 1774–1789. [Google Scholar] [CrossRef]
- Zhou, F.; De la Torre, F. Deformable Graph Matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, 23–28 June 2013; pp. 2922–2929. [Google Scholar]
- Tareen, S.A.K.; Saleem, Z. A comparative analysis of sift, surf, kaze, akaze, orb, and brisk. In Proceedings of the 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), Sukkur, Pakistan, 3–4 March 2018; pp. 1–10. [Google Scholar]
- Censi, A. An accurate closed-form estimate of ICP’s covariance. In Proceedings of the 2007 IEEE International Conference on Robotics and Automation, Roma, Italy, 10–14 April 2007; pp. 3167–3172. [Google Scholar]
- Nabiyev, V.V.; Yılmaz, S.; Günay, A.; Muzaffer, G.; Ulutaş, G. Shredded banknotes reconstruction using AKAZE points. Forensic Sci. Int. 2017, 278, 280–295. [Google Scholar] [CrossRef]
- Dellaert, F.; Kaess, M. Factor graphs for robot perception. Found. Trends® Robot. 2017, 6, 1–139. [Google Scholar] [CrossRef]
- Wang, J.; Chen, F.; Huang, Y.; McConnell, J.; Shan, T.; Englot, B. Virtual maps for autonomous exploration of cluttered underwater environments. IEEE J. Ocean. Eng. 2022, 47, 916–935. [Google Scholar] [CrossRef]
- Olson, E.B. Real-time correlative scan matching. In Proceedings of the 2009 IEEE International Conference on Robotics and Automation, Kobe, Japan, 12–17 May 2009; pp. 4387–4393. [Google Scholar]
- Carroll, S.B. Evo-devo and an expanding evolutionary synthesis: A genetic theory of morphological evolution. Cell 2008, 134, 25–36. [Google Scholar] [CrossRef]
Sensor Type | Model | Frequency | Features |
---|---|---|---|
Sonar | Oculus M1200d | 12.6 Hz | High resolution, high update rate |
Doppler Velocimeter | A50 | 10 Hz | Minimal size, max speed 2 m/s, ±0.1 cm/s accuracy |
Inertial Measurement Unit | BMI055 | 9.5 Hz | High stability |
Motion Capture System | NOKOV | 171 Hz | 2048 × 2048 resolution, 4.2 M pixels, 5.2 ms latency, 52° × 52° FOV |
GM SLAM/m | Bruce SLAM/m | |
---|---|---|
max | 0.360 | 0.944 |
mean | 0.118 | 0.189 |
median | 0.112 | 0.143 |
min | 0.014 | 0.007 |
rmse | 0.132 | 0.247 |
sse | 9.559 | 33.796 |
std | 0.059 | 0.159 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhuang, L.; Chen, X.; Lu, W.; Yan, Y. Graph Matching for Underwater Simultaneous Localization and Mapping Using Multibeam Sonar Imaging. J. Mar. Sci. Eng. 2024, 12, 1859. https://doi.org/10.3390/jmse12101859
Zhuang L, Chen X, Lu W, Yan Y. Graph Matching for Underwater Simultaneous Localization and Mapping Using Multibeam Sonar Imaging. Journal of Marine Science and Engineering. 2024; 12(10):1859. https://doi.org/10.3390/jmse12101859
Chicago/Turabian StyleZhuang, Lingfei, Xiaofeng Chen, Wenjie Lu, and Yiting Yan. 2024. "Graph Matching for Underwater Simultaneous Localization and Mapping Using Multibeam Sonar Imaging" Journal of Marine Science and Engineering 12, no. 10: 1859. https://doi.org/10.3390/jmse12101859
APA StyleZhuang, L., Chen, X., Lu, W., & Yan, Y. (2024). Graph Matching for Underwater Simultaneous Localization and Mapping Using Multibeam Sonar Imaging. Journal of Marine Science and Engineering, 12(10), 1859. https://doi.org/10.3390/jmse12101859