An Analysis of Research Trends for Using Artificial Intelligence in Cultural Heritage
Abstract
:1. Introduction
2. Research Method
3. Emerging Research Trends of Artificial Intelligence in Cultural Heritage
3.1. Classification
3.2. Computer Vision
3.3. 3D Reconstruction
3.4. Intangible Cultural Heritage
3.5. Recommender Systems
4. Conclusions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Mishra, M. Machine Learning Techniques for Structural Health Monitoring of Heritage Buildings: A State-of-the-art Review and Case Studies. J. Cult. Herit. 2021, 47, 227–245. [Google Scholar] [CrossRef]
- Yang, S.; Hou, M.; Li, S. Three-dimensional Point Cloud Semantic Segmentation for Cultural Heritage: A Comprehensive Review. Remote Sens. 2023, 15, 548. [Google Scholar] [CrossRef]
- Soleymani, A.; Jahangir, H.; Nehdi, M.L. Damage Detection and Monitoring in Heritage Masonry Structures: Systematic Review. Constr. Build. Mater. 2023, 397, 132402. [Google Scholar] [CrossRef]
- Barceló, J.; Del Castillo, F.; Kayikci, D.; Urbistondo, B. Neural Networks for Archaeological Classification and Typology: An overview. Eur. J. Post Class. Archaeol. 2022, 12, 7–32. [Google Scholar]
- Croce, V.; Caroti, G.; De Luca, L.; Piemonte, A.; Véron, P. Neural radiance fields (nerf): Review and Potential Applications to Digital Cultural Heritage. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2023, 48, 453–460. [Google Scholar] [CrossRef]
- Liu, Z. Literature Review on Image Restoration. J. Phys. Conf. Ser. 2022, 2386, 012041. [Google Scholar] [CrossRef]
- Rallis, I.; Voulodimos, A.; Bakalos, N.; Protopapadakis, E.; Doulamis, N.; Doulamis, A. Machine Learning for Intangible Cultural Heritage: A Review of Techniques on Dance Analysis. In Visual Computing for Cultural Heritage; Springer: Cham, Switzerland, 2020; pp. 103–119. [Google Scholar]
- Zhao, M.; Wu, X.; Liao, H.-T.; Liu, Y. Exploring Research Fronts and Topics of Big Data and Artificial Intelligence Application for Cultural Heritage and Museum Research. In IOP Conference Series: Materials Science and Engineering; IOP Publishing: Bristol, UK, 2020; p. 012036. [Google Scholar]
- Colavizza, G.; Blanke, T.; Jeurgens, C.; Noordegraaf, J. Archives and AI: An Overview of Current Debates and Future Perspectives. ACM J. Comput. Cult. Herit. (JOCCH) 2021, 15, 1–15. [Google Scholar] [CrossRef]
- Casillo, M.; Colace, F.; Gupta, B.B.; Lorusso, A.; Santaniello, D.; Valentino, C. The Role of AI in Improving Interaction with Cultural Heritage: An Overview. In Handbook of Research on AI and ML for Intelligent Machines and Systems; IGI Global: Hershey, PA, USA, 2024; pp. 107–136. [Google Scholar]
- Münster, S.; Ioannides, M. A Scientific Community of Digital Heritage in Time And Space. In Proceedings of the 2015 Digital Heritage, Granada, Spain, 28 September–2 October 2015; pp. 267–274. [Google Scholar]
- Zhang, Z.; Zou, Y. Research Hotspots and Trends in Heritage Building Information Modeling: A Review Based on Citespace Analysis. Humanit. Soc. Sci. Commun. 2022, 9, 394. [Google Scholar] [CrossRef]
- Das, S.; Mondal, S.; Puri, V.; Vrana, V. Structural Review of Relics Tourism by Text Mining and Machine Learning. J. Tour. Herit. Serv. Mark. (JTHSM) 2022, 8, 25–34. [Google Scholar]
- Ruiz-Rosero, J.; Ramírez-González, G.; Viveros-Delgado, J. Software Survey: Scientopy, a Scientometric Tool for Topics Trend Analysis in Scientific Publications. Scientometrics 2019, 121, 1165–1188. [Google Scholar] [CrossRef]
- Pranckutė, R. Web of Science (Wos) and Scopus: The Titans of Bibliographic Information in Today’s Academic World. Publications 2021, 9, 12. [Google Scholar] [CrossRef]
- Voinea, G.-D.; Gîrbacia, F.; Duguleană, M.; Boboc, R.G.; Gheorghe, C. Mapping the Emergent Trends in Industrial Augmented Reality. Electronics 2023, 12, 1719. [Google Scholar] [CrossRef]
- Munoz-Ausecha, C.; Ruiz-Rosero, J.; Ramirez-Gonzalez, G. RFID Applications and Security Review. Computation 2021, 9, 69. [Google Scholar] [CrossRef]
- Ćosović, M.; Janković, R. CNN Classification of the Cultural Heritage Images. In Proceedings of the 2020 19th International Symposium INFOTEH-JAHORINA (INFOTEH), East Sarajevo, Bosnia and Herzegovina, 18–20 March 2020; pp. 1–6. [Google Scholar]
- Abed, M.H.; Al-Asfoor, M.; Hussain, Z.M. Architectural Heritage Images Classification Using Deep Learning with CNN. In Proceedings of the 2nd International Workshop on Visual Pattern Extraction and Recognition for Cultural Heritage Understanding, Bari, Italy, 29 January 2020. [Google Scholar]
- Rehman, I.U.; Ali, Z.; Jan, Z.; Rashid, M.; Abbas, A.; Tariq, N. Deep Learning Empowered Classification of Augmented Cultural Heritage Images. In Proceedings of the VIPERC 2023: The 2nd International Conference on Visual Pattern Extraction and Recognition for Cultural Heritage Understanding, Zadar, Croatia, 25–26 September 2023. [Google Scholar]
- Gao, L.; Wu, Y.; Yang, T.; Zhang, X.; Zeng, Z.; Chan, C.K.D.; Chen, W. Research on Image Classification and Retrieval Using Deep Learning with Attention Mechanism on Diaspora Chinese Architectural Heritage in Jiangmen, China. Buildings 2023, 13, 275. [Google Scholar] [CrossRef]
- Wang, F.; Geng, S.; Zhang, D.; Zhou, M.; Nian, W.; Li, L. A Fine-Grained Classification Method of Thangka Image Based on Senet. In Proceedings of the 2022 International Conference on Cyberworlds (CW), Kanazawa, Japan, 27–29 September 2022; pp. 23–30. [Google Scholar]
- Wibawa, A.D.; Wicaksono, E.A.; Suryani, S.D.; Rumadi, R. Javanese Batik Image Classification using Self-Organizing Map. In Proceedings of the 2023 International Conference on Computer Science, Information Technology and Engineering (ICCoSITE), Jakarta, Indonesia, 16 February 2023; pp. 472–477. [Google Scholar]
- Prasomphan, S. Ensemble Classification Technique for Cultural Heritage Image. In Proceedings of the International Conference on Machine Learning and Intelligent Communications, Wuzhou, China, 17–18 November 2021; pp. 17–27. [Google Scholar]
- Muhathir, M.; Khairina, N.; Karenina, R.; Barus, I.; Ula, M.; Sahputra, I. Preserving Cultural Heritage Through AI: Developing LeNet Architecture for Wayang Image Classification. (IJACSA) Int. J. Adv. Comput. Sci. Appl. 2023, 14, 174. [Google Scholar] [CrossRef]
- Barucci, A.; Cucci, C.; Franci, M.; Loschiavo, M.; Argenti, F. A Deep Learning Approach to Ancient Egyptian Hieroglyphs Classification. IEEE Access 2021, 9, 123438–123447. [Google Scholar] [CrossRef]
- Barucci, A.; Canfailla, C.; Cucci, C.; Forasassi, M.; Franci, M.; Guarducci, G.; Guidi, T.; Loschiavo, M.; Picollo, M.; Pini, R. Ancient Egyptian Hieroglyphs Segmentation and Classification with Convolutional Neural Networks. In Proceedings of the International Conference Florence Heri-Tech: The Future of Heritage Science and Technologies, Florence, Italy, 16–18 May 2022; pp. 126–139. [Google Scholar]
- Fidatama, M.I.; Bimantoro, F.; Nugraha, G.S.; Irmawati, B.; Dwiyansaputra, R. Recognition of Bima Script Handwriting Patterns Using the Local Binary Pattern Feature Extraction Method and K-Nearest Neighbour Classification Method. In AIP Conference Proceedings; AIP Publishing: Melville, NY, USA, 2023. [Google Scholar]
- Mustiari, M.; Bimantoro, F.; Nugraha, G.S.; Husodo, A.Y. Bima Script Handwriting Pattern Recognition using Histogram of Oriented Gradients and Backpropagation Classification Method. In AIP Conference Proceedings; AIP Publishing: Melville, NY, USA, 2023. [Google Scholar]
- Navarro, P.; Cintas, C.; Lucena, M.; Fuertes, J.M.; Delrieux, C.; Molinos, M. Learning Feature Representation of Iberian Ceramics with Automatic Classification Models. J. Cult. Herit. 2021, 48, 65–73. [Google Scholar] [CrossRef]
- Cascone, L.; Dondi, P.; Lombardi, L.; Narducci, F. Automatic Classification of Fresco Fragments: A Machine and Deep Learning Study. In Proceedings of the International Conference on Image Analysis and Processing, Lecce, Italy, 23–27 May 2022; pp. 701–712. [Google Scholar]
- Obeso, A.M.; Benois-Pineau, J.; Vázquez, M.G.; Acosta, A.R. Saliency-Based Selection of Visual Content for Deep Convolutional Neural Networks: Application to Architectural Style Classification. Multimed. Tools Appl. 2019, 78, 9553–9576. [Google Scholar] [CrossRef]
- Taoufiq, S.; Nagy, B.; Benedek, C. Hierarchynet: Hierarchical CNN-based Urban Building Classification. Remote Sens. 2020, 12, 3794. [Google Scholar] [CrossRef]
- Marafini, F.; Betti, M.; Bartoli, G.; Zini, G.; Barontini, A.; Mendes, N. A Proposal of Classification for Machine-Learning Vibration-Based Damage Identification Methods. Mater. Res. Proc. 2023, 26, 593–598. [Google Scholar]
- Mehta, S.; Kukreja, V.; Gupta, A. Exploring the Efficacy of CNN and SVM Models for Automated Damage Severity Classification in Heritage Buildings. In Proceedings of the 2023 Second International Conference on Augmented Intelligence and Sustainable Systems (ICAISS), Trichy, India, 23–25 August 2023; pp. 252–257. [Google Scholar]
- Lee, J.; Yu, J. Automatic Surface Damage Classification Developed Based on Deep Learning for Wooden Architectural Heritage. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 2023, X-M-1-2023, 151–157. [Google Scholar] [CrossRef]
- Roy, P.S.; Kukreja, V.; Jain, V.; Vats, S. Classification of Defective Intensity Levels of Paint in Heritage Buildings using the CNN-SVM Technique. In Proceedings of the 2023 5th International Conference on Inventive Research in Computing Applications (ICIRCA), Coimbatore, India, 3–5 August 2023; pp. 17–22. [Google Scholar]
- Rodrigues, F.; Cotella, V.; Rodrigues, H.; Rocha, E.; Freitas, F.; Matos, R. Application of Deep Learning Approach for the Classification of Buildings’ Degradation State. In A BIM Methodology. Appl. Sci. 2022, 12, 7403. [Google Scholar] [CrossRef]
- Fu, Q.; Hou, M.; Hua, W. Based on Improved Pointnet Automatic Classification Method of Grotto Temple Statues. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 2023, X-M-1-2023, 87–92. [Google Scholar] [CrossRef]
- Pepe, M.; Alfio, V.S.; Costantino, D.; Scaringi, D. Data for 3D Reconstruction and Point Cloud Classification using Machine Learning in Cultural Heritage Environment. Data Brief 2022, 42, 108250. [Google Scholar] [CrossRef]
- Grilli, E.; Remondino, F. Classification of 3D Digital Heritage. Remote Sens. 2019, 11, 847. [Google Scholar] [CrossRef]
- Croce, V.; Manuel, A.; Caroti, G.; Piemonte, A.; De Luca, L.; Véron, P. Semi-Automatic Classification of Digital Heritage on the Aïoli Open Source 2D/3D Annotation Platform via Machine Learning and Deep Learning. J. Cult. Herit. 2023, 62, 187–197. [Google Scholar] [CrossRef]
- Rani, C.J.; Devarakonda, N. An Effectual Classical Dance Pose Estimation and Classification System Employing Convolution Neural Network–Long Shortterm Memory (CNN-LSTM) Network for Video Sequences. Microprocess. Microsyst. 2022, 95, 104651. [Google Scholar] [CrossRef]
- Malavath, P.; Devarakonda, N. Natya Shastra: Deep Learning for Automatic Classification of Hand Mudra in Indian Classical Dance Videos. Rev. D’Intell. Artif. 2023, 37, 689. [Google Scholar] [CrossRef]
- Tiwari, R.G.; Gautam, V.; Trivedi, N.K. Heritage of India: Advanced Monuments Classification using Artificial Intelligence. In Proceedings of the 2023 3rd International Conference on Computing and Information Technology (ICCIT), Tabuk, Saudi Arabia, 13–14 September 2023; pp. 206–210. [Google Scholar]
- Chen, Q.; Zhao, W.; Wang, Q.; Zhao, Y. The Sustainable Development of Intangible Cultural Heritage with AI: Cantonese Opera Singing Genre Classification Based On Cogcnet Model In China. Sustainability 2022, 14, 2923. [Google Scholar] [CrossRef]
- Castellano, G.; Lella, E.; Vessio, G. Visual Link Retrieval and Knowledge Discovery in Painting Datasets. Multimed. Tools Appl. 2021, 80, 6599–6616. [Google Scholar] [CrossRef]
- Castellano, G.; Vessio, G. A brief Overview of Deep Learning Approaches to Pattern Extraction and Recognition in Paintings and Drawings. In Proceedings of the International Conference on Pattern Recognition, Shanghai, China, 15–17 October 2021; pp. 487–501. [Google Scholar]
- Marinescu, M.-C.; Reshetnikov, A.; López, J.M. Improving Object Detection in Paintings Based on Time Contexts. In Proceedings of the 2020 International Conference on Data Mining Workshops (ICDMW), Sorrento, Italy, 17–20 November 2020; pp. 926–932. [Google Scholar]
- Pathak, R.; Saini, A.; Wadhwa, A.; Sharma, H.; Sangwan, D. An Object Detection Approach for Detecting Damages in Heritage Sites Using 3-D Point Clouds and 2-D Visual Data. J. Cult. Herit. 2021, 48, 74–82. [Google Scholar] [CrossRef]
- Bacharidis, K.; Sarri, F.; Ragia, L. 3D Building Façade Reconstruction using Deep Learning. ISPRS Int. J. Geo-Inf. 2020, 9, 322. [Google Scholar] [CrossRef]
- Payntar, N.D.; Hsiao, W.-L.; Covey, R.A.; Grauman, K. Learning Patterns of Tourist Movement and Photography from Geotagged Photos at Archaeological Heritage Sites in Cuzco, Peru. Tour. Manag. 2021, 82, 104165. [Google Scholar] [CrossRef]
- Ferrato, A.; Limongelli, C.; Mezzini, M.; Sansonetti, G. Using Deep Learning for Collecting Data about Museum Visitor Behavior. Appl. Sci. 2022, 12, 533. [Google Scholar] [CrossRef]
- Fenu, G.; Medvet, E.; Panfilo, D.; Pellegrino, F.A. Mosaic Images Segmentation using U-net. In Proceedings of the 9th International Conference on Pattern Recognition Applications and Methods, Valletta, Malta, 22–24 February 2020; pp. 485–492. [Google Scholar]
- Condorelli, F. Processing Historical Photographs and Film Footage with Photogrammetry and Artificial Intelligence for Cultural Heritage Documentation and Virtual Reconstruction. ELCVIA Electron. Lett. Comput. Vis. Image Anal. 2020, 19, 11–16. [Google Scholar] [CrossRef]
- Yang, X.; Zheng, L.; Chen, Y.; Feng, J.; Zheng, J. Recognition of Damage Types of Chinese Gray-Brick Ancient Buildings Based on Machine Learning—Taking the Macau World Heritage Buffer Zone as an example. Atmosphere 2023, 14, 346. [Google Scholar] [CrossRef]
- Lee, S.-Y.; Cho, H.-H. Damage Detection and Safety Diagnosis for Immovable Cultural Assets Using Deep Learning Framework. In Proceedings of the 2023 25th International Conference on Advanced Communication Technology (ICACT), Pyeongchang, Republic of Korea, 19–22 February 2023; pp. 310–313. [Google Scholar]
- Kwon, D.; Yu, J. Automatic Damage Detection of Stone Cultural Property Based on Deep Learning Algorithm. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2019, 42, 639–643. [Google Scholar] [CrossRef]
- Li, J.; Wang, H.; Deng, Z.; Pan, M.; Chen, H. Restoration of Non-Structural Damaged Murals in Shenzhen Bao’an Based on a Generator–Discriminator Network. Herit. Sci. 2021, 9, 6. [Google Scholar] [CrossRef]
- Kumar, P.; Gupta, V. Restoration of Damaged Artworks Based on a Generative Adversarial Network. Multimed. Tools Appl. 2023, 82, 40967–40985. [Google Scholar] [CrossRef]
- Castiello, M.E.; Tonini, M. An Explorative Application of Random Forest Algorithm for Archaeological Predictive Modeling. A Swiss Case Study. J. Comput. Appl. Archaeol. 2021, 4, 110–125. [Google Scholar] [CrossRef]
- Orengo, H.A.; Garcia-Molsosa, A. A Brave New World for Archaeological Survey: Automated Machine Learning-Based Potsherd Detection using High-Resolution Drone Imagery. J. Archaeol. Sci. 2019, 112, 105013. [Google Scholar] [CrossRef]
- Wang, Y.; Shi, X.; Oguchi, T. Archaeological Predictive Modeling Using Machine Learning and Statistical Methods for Japan and China. ISPRS Int. J. Geo-Inf. 2023, 12, 238. [Google Scholar] [CrossRef]
- Argyrou, A.; Agapiou, A. A Review of Artificial Intelligence and Remote Sensing For Archaeological Research. Remote Sens. 2022, 14, 6000. [Google Scholar] [CrossRef]
- Stoean, R.; Ionescu, L.; Stoean, C.; Boicea, M.; Atencia, M.; Joya, G. A Deep Learning-Based Surrogate for the Xrf Approximation of Elemental Composition within Archaeological Artefacts before Restoration. Procedia Comput. Sci. 2021, 192, 2002–2011. [Google Scholar] [CrossRef]
- Jain, N.; Sierra-Múnera, A.; Ehmueller, J.; Krestel, R. Generation of Training Data for Named Entity Recognition of Artworks. Semant. Web 2023, 14, 239–260. [Google Scholar] [CrossRef]
- Moradi, M.; Ghorbani, R.; Sfarra, S.; Tax, D.M.; Zarouchas, D. A Spatiotemporal Deep Neural Network useful for Defect Identification and Reconstruction of Artworks using Infrared Thermography. Sensors 2022, 22, 9361. [Google Scholar] [CrossRef] [PubMed]
- Angheluţă, L.; Chiroşca, A. Physical Degradation Detection on Artwork Surface Polychromies using Deep Learning Models. Rom. Rep. Phys. 2020, 72, 805. [Google Scholar]
- El Makhfi, N. Handwritten Text Segmentation Approach in Historical Arabic Documents. In Embedded Systems and Artificial Intelligence: Proceedings of ESAI 2019, Fez, Morocco, 2–3 May 2019; Springer: Singapore, 2020; pp. 645–654. [Google Scholar]
- Ferro, S.; Pelillo, M.; Traviglia, A. AI-Assisted Digitalisation of Historical Documents. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2023, 48, 557–562. [Google Scholar] [CrossRef]
- Philips, J.; Tabrizi, N. Historical Document Processing: A Survey of Techniques, Tools, and Trends. In Proceedings of the 12th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2020)—Volume 1: KDIR, Online, 2–4 November 2020; pp. 341–349. [Google Scholar]
- Ostertag, C.; Beurton-Aimar, M. Using Graph Neural Networks to Reconstruct Ancient Documents. In Proceedings of the Pattern Recognition. ICPR International Workshops and Challenges, Virtual Event, 10–15 January 2021; Proceedings, Part VII. Springer International Publishing: Berlin/Heidelberg, Germany, 2021; pp. 39–53. [Google Scholar]
- Babić, R.J.; Amelio, A.; Draganov, I.R. Writer Identification From Historical Documents Using Ensemble Deep Learning Transfer Models. In Proceedings of the 2022 21st International Symposium INFOTEH-JAHORINA (INFOTEH), East Sarajevo, Bosnia and Herzegovina, 16–18 March 2022; pp. 1–5. [Google Scholar]
- Ángel García-Calderón, M.; García-Hernndez, R.; Ledeneva, Y. An Unsupervised Lower-Baseline Localization Method Based on Writing Style Features for Historical Documents. J. Intell. Fuzzy Syst. 2020, 39, 2509–2520. [Google Scholar] [CrossRef]
- Kaneko, H.; Yoshizu, Y.; Ishibashi, R.; Meng, L. An Attempt at Zero-shot Ancient Documents Restoration Based on Diffusion Models. In Proceedings of the 2023 International Conference on Advanced Mechatronic Systems (ICAMechS), Melbourne, Australia, 4–7 September 2023; pp. 1–6. [Google Scholar]
- Colutto, S.; Kahle, P.; Guenter, H.; Muehlberger, G. Transkribus. A Platform for Automated Text Recognition and Searching of Historical Documents. In Proceedings of the 2019 15th International Conference on eScience (eScience), San Diego, CA, USA, 24–27 September 2019; pp. 463–466. [Google Scholar]
- Jaillant, L.; Caputo, A. Unlocking Digital Archives: Cross-Disciplinary Perspectives on AI and Born-Digital Data. AI Soc. 2022, 37, 823–835. [Google Scholar] [CrossRef]
- Tzouganatou, A. Openness and Privacy in Born-Digital Archives: Reflecting the Role of AI Development. AI Soc. 2022, 37, 991–999. [Google Scholar] [CrossRef]
- Schleider, T.; Troncy, R. Zero-Shot Information Extraction to Enhance a Knowledge Graph Describing Silk Textiles. In Proceedings of the 5th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature, Online, 11 November 2021; pp. 138–146. [Google Scholar]
- Stoean, C.; Bacanin, N.; Stoean, R.; Ionescu, L.; Alecsa, C.; Hotoleanu, M.; Atencia, M.; Joya, G. On Using Perceptual Loss within the U-Net Architecture for the Semantic Inpainting of Textile Artefacts with Traditional Motifs. In Proceedings of the 2022 24th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), Hagenberg/Linz, Austria, 12–15 September 2022; pp. 276–283. [Google Scholar]
- Ilies, D.C.; Zlatev, Z.; Ilies, A.; Zharas, B.; Pantea, E.; Hodor, N.; Indrie, L.; Turza, A.; Taghiyari, H.R.; Caciora, T. Interdisciplinary Research to Advance Digital Imagery and Natural Compounds for Eco-Cleaning and for Preserving Textile Cultural Heritage. Sensors 2022, 22, 4442. [Google Scholar] [CrossRef]
- Liu, M.; Zhou, B. Innovative Design of Chinese Traditional Textile Patterns Based on Conditional Generative Adversarial Network. In Proceedings of the International Conference on Human-Computer Interaction, Virtual Event, 26 June–1 July 2022; pp. 234–245. [Google Scholar]
- Ch’ng, E.; Feng, P.; Yao, H.; Zeng, Z.; Cheng, D.; Cai, S. Balancing Performance and Effort in Deep Learning via the Fusion of Real and Synthetic Cultural Heritage Photogrammetry Training Sets. In Proceedings of the ICAART (1), Online, 4–6 February 2021; pp. 611–621. [Google Scholar]
- Condorelli, F. Image Retrieval for 3D Modelling of Architecture using AI and Photogrammetry. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2023, 48, 441–446. [Google Scholar] [CrossRef]
- Niu, W.; Huang, X.; Jin, J.; Mao, Z.; Gong, Y.; Xu, J.; Zhao, J. Recognition Method of the main Object of Three-Dimensional Photogrammetric Modeling of Cultural Relics. Natl. Remote Sens. Bull. 2022, 25, 2409–2420. [Google Scholar] [CrossRef]
- Grilli, E.; Farella, E.M.; Torresani, A.; Remondino, F. Geometric Features Analysis for the Classification of Cultural Heritage Point Clouds. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2019, 42, 541–548. [Google Scholar] [CrossRef]
- Gujski, L.; di Filippo, A.; Limongiello, M. Machine Learning Clustering for Point Clouds Optimisation via Feature Analysis in Cultural Heritage. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2022, 46, 245–251. [Google Scholar] [CrossRef]
- Pellis, E.; Murtiyoso, A.; Masiero, A.; Tucci, G.; Betti, M.; Grussenmeyer, P. An Image-based Deep Learning Workflow for 3D Heritage Point Cloud Semantic Segmentation. In Proceedings of the 9th International Workshop 3D-ARCH “3D Virtual Reconstruction and Visualization of Complex Architectures”, Mantua, Italy, 2–4 March 2022; pp. 426–434. [Google Scholar]
- Cao, Y.; Scaioni, M. 3DLEB-Net: Label-Efficient Deep Learning-Based Semantic Segmentation of Building Point Clouds at Lod3 Level. Appl. Sci. 2021, 11, 8996. [Google Scholar] [CrossRef]
- Cao, Y.; Teruggi, S.; Fassi, F.; Scaioni, M. A Comprehensive Understanding f Machine Learning and Deep Learning Methods for 3d Architectural Cultural Heritage Point Cloud Semantic Segmentation. In Proceedings of the Italian Conference on Geomatics and Geospatial Technologies, Genova, Italy, 18–20 December 2022; pp. 329–341. [Google Scholar]
- Pellis, E.; Masiero, A.; Tucci, G.; Betti, M.; Grussenmeyer, P. Assembling an Image and Point Cloud Dataset for Heritage Building Semantic Segmentation. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2021, 46, 539–546. [Google Scholar] [CrossRef]
- Musicco, A.; Galantucci, R.A.; Bruno, S.; Verdoscia, C.; Fatiguso, F. Automatic Point Cloud Segmentation for the Detection of Alterations on Historical Buildings Through an Unsupervised and Clustering-Based Machine Learning Approach. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 2021, 2, 129–136. [Google Scholar] [CrossRef]
- Malinverni, E.S.; Pierdicca, R.; Paolanti, M.; Martini, M.; Morbidoni, C.; Matrone, F.; Lingua, A. Deep Learning for Semantic Segmentation of 3D Point Cloud. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2019, 42, 735–742. [Google Scholar] [CrossRef]
- Haznedar, B.; Bayraktar, R.; Ozturk, A.E.; Arayici, Y. Implementing Pointnet for Point Cloud Segmentation in the Heritage Context. Herit. Sci. 2023, 11, 2. [Google Scholar] [CrossRef]
- Galanakis, D.; Maravelakis, E.; Pocobelli, D.P.; Vidakis, N.; Petousis, M.; Konstantaras, A.; Tsakoumaki, M. SVD-based Point Cloud 3D Stone by Stone Segmentation for Cultural Heritage Structural Analysis–The Case of the Apollo Temple at Delphi. J. Cult. Herit. 2023, 61, 177–187. [Google Scholar] [CrossRef]
- Pierdicca, R.; Paolanti, M.; Matrone, F.; Martini, M.; Morbidoni, C.; Malinverni, E.S.; Frontoni, E.; Lingua, A.M. Point Cloud Semantic Segmentation using a Deep Learning Framework for Cultural Heritage. Remote Sens. 2020, 12, 1005. [Google Scholar] [CrossRef]
- Foryś, P.; Sitnik, R.; Markiewicz, J.; Bunsch, E. Fast Adaptive Multimodal Feature Registration (FAMFR): An Effective High-Resolution Point Clouds Registration Workflow for Cultural Heritage Interiors. Herit. Sci. 2023, 11, 190. [Google Scholar] [CrossRef]
- Saiti, E.; Danelakis, A.; Theoharis, T. Cross-Time Registration of 3D Point Clouds. Comput. Graph. 2021, 99, 139–152. [Google Scholar] [CrossRef]
- Tabib, R.A.; Kulkarni, S.; Kagalkar, A.; Hurakadli, V.; Ganapule, A.; Dhanakshirur, R.R.; Mudenagudi, U. Deep Learning-Based Filtering of Images For 3d Reconstruction of Heritage Sites. In Digital Techniques for Heritage Presentation and Preservation; Springer: Cham, Switzerland, 2021; pp. 147–156. [Google Scholar]
- Wang, Z.; Xiong, H. Analysis of the Application of Deep Learning in Model Reconstruction of Ancient Buildings. Adv. Multimed. 2022, 2022, 4273937. [Google Scholar] [CrossRef]
- Dou, J.; Qin, J.; Jin, Z.; Li, Z. Knowledge Graph Based on Domain Ontology and Natural Language Processing Technology for Chinese Intangible Cultural Heritage. J. Vis. Lang. Comput. 2018, 48, 19–28. [Google Scholar] [CrossRef]
- Bakalos, N.; Rallis, I.; Doulamis, N.; Doulamis, A.; Protopapadakis, E.; Voulodimos, A. Choreographic Pose Identification Using Convolutional Neural Networks. In Proceedings of the 2019 11th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games), Vienna, Austria, 4–6 September 2019; pp. 1–7. [Google Scholar]
- Serra, L. Preservation of Mediterranean Intangible Cultural Heritage through Virtual Gaming and Informatics: The Case of Sardinian Mùrra. In Proceedings of the 14th International Multi-Conference on Society, Cybernetics and Informatics (IMSCI 2020), Online, 13–16 September 2020; pp. 94–98. [Google Scholar]
- Fugini, M.; Finocchi, J.; Rossi, E. Semantic Adaptive Enrichment of Cartography for Intangible Cultural Heritage and Citizen Journalism. In Proceedings of the Future of Information and Communication Conference, Berlin, Germany, 4–5 April 2022; pp. 173–185. [Google Scholar]
- Sotiropoulos, D.N.; Tsihrintzis, G.A.; Virvou, M.; Tsichrintzi, E.-A. Machine Learning in Intangible Cultural Analytics: The Case of Greek Songs’ Lyrics. In Proceedings of the 2021 IEEE 33rd International Conference on Tools with Artificial Intelligence (ICTAI), Washington, DC, USA, 1–3 November 2021; pp. 299–305. [Google Scholar]
- Do, T.-N.; Pham, N.-K.; Nguyen, H.-H.; Tabia, K.; Benferhat, S. Stacking of Svms for Classifying Intangible Cultural Heritage Images. In Proceedings of the International Conference on Computer Science, Applied Mathematics and Applications, Hanoi, Vietnam, 19–20 December 2019; pp. 186–196. [Google Scholar]
- Zhang, X.; Jin, Y. A Method of Protecting Sensitive Information in Intangible Cultural Heritage Communication Network Based on Machine Learning. In Proceedings of the International Conference on Machine Learning for Cyber Security, Guangzhou, China, 2–4 December 2022; pp. 214–227. [Google Scholar]
- Zhu, L.; Pang, T. Research on Digital Platform Technology of Intangible Cultural Heritage in Beijing Section of Great Wall Cultural Belt. In Proceedings of the 2022 International Conference on Culture-Oriented Science and Technology (CoST), Lanzhou, China, 18–21 August 2022; pp. 426–430. [Google Scholar]
- Huang, L.; Song, Y. Intangible Cultural Heritage Management using Machine Learning Model: A Case Study of Northwest Folk Song Huaer. Sci. Program. 2022, 2022, 1383520. [Google Scholar] [CrossRef]
- Zhao, H. The Database Construction of Intangible Cultural Heritage Based on Artificial Intelligence. Math. Probl. Eng. 2022, 2022, 8576002. [Google Scholar] [CrossRef]
- Li, Q. Intelligent Intangible Cultural Heritage Innovation Platform under the Background of Big Data and Virtual Systems. In Proceedings of the 2022 Second International Conference on Artificial Intelligence and Smart Energy (ICAIS), Coimbatore, India, 23–25 February 2022; pp. 560–563. [Google Scholar]
- Yang, Z.; Jia, X.; Zhang, X.; Tang, J. Interpretable Neural Symbol Learning Methods to Fuse Deep Learning Representation and Knowledge Graph: Zhejiang Cuisine Recipe Intangible Cultural Heritage Use Case. In Design Studies and Intelligence Engineering; IOS Press: Amsterdam, The Netherlands, 2023; pp. 53–61. [Google Scholar]
- Huang, W.; Zheng, J.; Chen, W. The Construction of Intelligent Education for Intangible Cultural Heritage into the Classroom. In Proceedings of the The International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery, Xi’an, China, 1–3 August 2020; pp. 1172–1181. [Google Scholar]
- Xu, Z.; Zou, D. Big Data Analysis Research on the Deep Integration of Intangible Cultural Heritage Inheritance and Art Design Education in Colleges and Universities. Mob. Inf. Syst. 2022, 2022, 1172405. [Google Scholar] [CrossRef]
- Pistola, T.; Diplaris, S.; Stentoumis, C.; Stathopoulos, E.A.; Loupas, G.; Mandilaras, T.; Kalantzis, G.; Kalisperakis, I.; Tellios, A.; Zavraka, D. Creating Immersive Experiences Based on Intangible Cultural Heritage. In Proceedings of the 2021 IEEE International Conference on Intelligent Reality (ICIR), Piscataway, NJ, USA, 12–13 May 2021; pp. 17–24. [Google Scholar]
- Cui, J.; Fu, L. Multimedia Display of Wushu Intangible Cultural Heritage Based on Interactive System and Artificial Intelligence. Soft Comput. 2023, 27, 1–9. [Google Scholar] [CrossRef]
- Hu, R. Research on the Digital Protection Technology of Intangible Cultural Heritage Information under Computer Artificial Intelligence. In Proceedings of the 2022 International Conference on Computers, Information Processing and Advanced Education (CIPAE), Ottawa, ON, Canada, 26–28 August 2022; pp. 92–96. [Google Scholar]
- Yu, X.; Shan, W.; Ding, H.; Li, B. Research on Intangible Cultural Heritage Amusement Space Design from the Perspective of Artificial Intelligence. In Proceedings of the 2021 2nd International Conference on Intelligent Design (ICID), Xi’an, China, 19 October 2021; pp. 203–207. [Google Scholar]
- Pavlidis, G. Recommender Systems, Cultural Heritage Applications, and The Way Forward. J. Cult. Herit. 2019, 35, 183–196. [Google Scholar] [CrossRef]
- Dam, N.A.K.; Le Dinh, T. A Literature Review of Recommender Systems for the Cultural Sector. In Proceedings of the 22nd International Conference on Enterprise Information Systems (ICEIS 2020), Prague, Czech Republic, 5–7 May 2020; Volume 1, pp. 715–726. [Google Scholar]
- Ferrato, A.; Limongelli, C.; Mezzini, M.; Sansonetti, G. The META4RS Proposal: Museum Emotion and Tracking Analysis for Recommender Systems. In Proceedings of the Adjunct Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization, Barcelona, Spain, 4–7 July 2022; pp. 406–409. [Google Scholar]
- Trichopoulos, G. Large Language Models for Cultural Heritage. In Proceedings of the 2nd International Conference of the ACM Greek SIGCHI Chapter, Athens, Greece, 27–28 September 2023; pp. 1–5. [Google Scholar]
- Nafis, F.; Al Fararni, K.; Yahyaouy, A.; Aghoutane, B. An Approach Based on Machine Learning Algorithms for the Recommendation of Scientific Cultural Heritage Objects. Int. J. Adv. Comput. Sci. Appl. 2021, 12, 230. [Google Scholar] [CrossRef]
- Colace, F.; D’Arienzo, M.P.; Lorusso, A.; Lombardi, M.; Santaniello, D.; Valentino, C. A Novel Context Aware Paths Recommendation Approach for the Cultural Heritage Enhancement. In Proceedings of the 2023 IEEE International Conference on Smart Computing (SMARTCOMP), Nashville, TN, USA, 26–30 June 2023; pp. 273–278. [Google Scholar]
- Casillo, M.; Conte, D.; Lombardi, M.; Santaniello, D.; Valentino, C. Recommender System For Digital Storytelling: A Novel Approach to Enhance Cultural Heritage. In Proceedings of the Pattern Recognition. ICPR International Workshops and Challenges, Virtual Event,, 10–15 January 2021; Proceedings, Part VII. Springer International Publishing: Berlin/Heidelberg, Germany, 2021; pp. 304–317. [Google Scholar]
- Su, X.; Sperlì, G.; Moscato, V.; Picariello, A.; Esposito, C.; Choi, C. An Edge Intelligence Empowered Recommender System Enabling Cultural Heritage Applications. IEEE Trans. Ind. Inform. 2019, 15, 4266–4275. [Google Scholar] [CrossRef]
- Arrieta, A.B.; Díaz-Rodríguez, N.; Ser, J.D.; Bennetot, A.; Tabik, S.; Barbado, A.; Garcia, S.; Gil-Lopez, S.; Molina, D.; Benjamins, R.; et al. Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI. Inf. Fusion 2019, 58, 82–115. [Google Scholar] [CrossRef]
Pos | Author Keywords | Total | AGR | ADY | PDLY | h-Index |
---|---|---|---|---|---|---|
1 | Classification | 124 | 7.5 | 32.5 | 52.4 | 19 |
2 | Computer vision | 100 | 3.5 | 24.5 | 49.0 | 14 |
3 | 3D Reconstruction | 81 | 6.0 | 18.5 | 45.7 | 15 |
4 | Intangible Cultural Heritage | 32 | 4.0 | 10.5 | 65.6 | 4 |
5 | Recommender systems | 16 | −1.0 | 2.0 | 25.0 | 3 |
Reference | Brief Summary of the Method | Data | Key Findings | Challenges |
---|---|---|---|---|
[21] | Enhance the classification and retrieval of diaspora Chinese architectural images in Jiangmen, utilizing a Convolutional Neural Network with an Attention Mechanism (CNNAR Framework) | 5073 images of diaspora Chinese buildings, auxiliary datasets from JMI architectural heritage images, Paris500K and Corel5K | CNNAR model achieved a mean average precision of 76.6% and high classification accuracy of 98.3% in JMI datasets | Managing variations in lighting; occlusions; redundancy; and complex feature extraction across architectural styles |
[26] | Developing a custom architecture called Glyphnet to classify ancient Egyptian hieroglyphs | Two labeled datasets comprising over 4000 images of hieroglyphs | Glyphnet outperformed existing CNN models (ResNet-50, Inception-v3, Xception) in accuracy and computational efficiency | Historical complexity of hieroglyphs; effective methods for semi-automated recognition |
[30] | Classification of Iberian ceramics using automated methods, including transfer learning with ResNet-18 and various classifiers to optimize feature extraction | 1282 labeled images of ceramic vessels | The study achieved high accuracy, improving classification flexibility and accuracy | Variability among image quality of pottery vessels, limited sample sizes affecting model training; managing overfitting |
[39] | Explore an improved PointNet network for the automatic classification of grotto temple statues | 3D point cloud data from the Eighteen Grottoes of Yungang, captured with 3D laser scanners | Achieved overall classification accuracy of 89.73%, outperforming random forest classifiers | Managing irregular statue positions; rough edge detail resolution |
[41] | Using of both supervised and unsupervised ML strategies for semi-automatic classification of digital heritage on the Aïoli platform. | Digital models from photogrammetric surveys and 2D/3D datasets | Improvements in classification accuracy and the successful transfer of annotations across formats | Difficulties with aligning 2D images with 3D point clouds; managing extensive datasets |
Reference | Brief Summary of the Method | Data | Key Findings | Challenges |
---|---|---|---|---|
[56] | Identification of damage types in Chinese gray-brick ancient buildings within the Macau World Heritage Buffer Zone using the YOLOv4 ML model | 1000 labeled images covering five specific damaged gray bricks | Detected damage with 85.7% accuracy, but misidentified some stains as missing bricks and struggled with low-contrast conditions. | Limited training data, ambiguity in labeling, and the effects of environmental conditions on detection accuracy |
[59] | Restore non-structurally damaged murals in Bao’an District using a Generator-Discriminator Network (GAN) approach, focusing on enhancing texture, color, and structural continuity | 137 murals images for training and 22 damaged murals for restoration from Shenzhen Bao’ | Improvements in restoration quality, achieving an average Peak Signal-to-Noise Ratio of 34.36 and a Structural Similarity Index Measure of 0.91 | Predicting missing details; managing noise; preserving original details during reconstruction |
[61] | Assess the spatial probability of Roman settlements in the Canton of Zurich using the Random Forest (RF) algorithm for archaeological predictive modeling | 227 occurrences of Roman settlements, extracted from a larger collection of 5812 entries, and incorporated geo-environmental factors | Findings indicated an Area Under the Curve (AUC) of 0.72, with significant correlations between agricultural suitability and site locations | Data quality issues; the need for spatial resolution standardization |
[70] | Develop AI-assisted digitization of historical documents by improving automatic handwriting recognition through transfer learning and fine-tuning techniques | IAM dataset for pre-training; various historical datasets including Saint Gall, Parzival, Washington, and Specchieri Marigold | Achieving a Character Error Rate of less than 10% is feasible with sufficient training samples | Noise from text deterioration; limited access to extensive labeled data |
[75] | Restore ancient documents using a zero-shot approach with the Denoising Diffusion Restoration Model (DDRM), and noise masking | Kuzushi-ji dataset consists of 4328 classes and 1,086,236 Japanese ancient characters | Effective restoration without retraining; PSNR scores indicating performance improvements | Dependency on pre-trained models; handling various degradations |
Reference | Brief Summary of the Method | Data | Key Findings | Challenges |
---|---|---|---|---|
[92] | Automate point cloud segmentation for detecting alterations in historical buildings using erarchical clustering in the HSV color space and RANSAC for shape identification | Point clouds from photogrammetric data of three architectural case studies | Effectiveness of HSV for segmentation, achieving accurate classifications of surface conditions | Adapting algorithms for diverse architectural features; accurate clustering; capturing complex masonry surface pathologies |
[94] | Segment historical structures from 3D point cloud data using PointNet | Point clouds from heritage buildings in Gaziantep, comprising 19 buildings and 140 rooms | Improved segmentation accuracy, achieving up to 91.20% when restitution data was included | High-quality data labeling; management of deformed building elements |
[95] | Improve stone-level segmentation of the Apollo Temple’s point cloud using Singular Value Decomposition (SVD) for structural analysis and minimal user intervention. | Point cloud data from Terrestrial Laser Scanning (TLS) and UAVs | Enables segmentation without prior shape knowledge, 92% accuracy in identifying joints in multi-slab columns | Accurate boundary detection; local geometry reconstruction |
[97] | Development of the Fast Adaptive Multimodal Feature Registration (FAMFR) workflow for effective registration of high-resolution point clouds in cultural heritage interiors | Point clouds from 3D scanners, photogrammetry, and LiDAR from complex decorative surfaces at the Museum of King Jan III’s Palace | FAMFR significantly improve registration accuracy, showcasing its efficacy in handling intricate geometric and decorative surfaces | Complex alignment of multiple point clouds; noise from reflective surfaces |
Reference | Brief Summary of the Method | Data | Key Findings | Challenges |
---|---|---|---|---|
[105] | Develop the MUSILYAN tool for automated analysis of Greek song lyrics through thematic organization and semantic categorization using k-means clustering | 447 songs by Costas Virvos filtered to identify 1250 unique terms | Revealed significant emotional and semantic information in lyrics | Integrating diverse data forms for audio analysis, automation of thematic organization |
[109] | Using recurrent neural network to manage and recognize the Huaer Northwest China folk music | Lyrics and audio resources collected through Python web crawlers | Enhancing dataset accessibility; effective recognition algorithms | Data collection due to regional diversity, difficulties in Chinese word segmentation, and |
[112] | Develop RF-YOLOv5 interpretable neural-symbol learning method to enhance the interpretability of DL models for recipe recommendations in Zhejiang cuisine | 3005 images of food items across 15 ingredient categories | Improved Zhejiang cuisine recipe recommendation system aligning target detection with a knowledge graph;identification accuracy of 93.61% | Model complexity; the need for extensively annotated datasets; data cleaning challenges |
Reference | Brief Summary of the Method | Data | Key Findings | Challenges |
---|---|---|---|---|
[121] | Development of the META4RS system capable of tracking the location of the visitor by using DL; assessing the emotional reaction to the artworks observed through simple badges and off-the-shelf cameras. | Visitor positional data and emotional reactions | Personalized recommendations can enhance the visitor experience | Ensuring visitor anonymity; accuracy of emotion inference; system integration |
[122] | Enhance museum experiences using Generative Pretrained Transformer (GPT-4) for personalized guidance and storytelling through the MAGICAL project. | Fictional museum MMMA | Provide adapted information, recommendations, and even immersive storytelling based on visitor input | Technological limitations of smart glasses; handling diverse visitor knowledge levels; inaccuracies in AI-generated responses |
[123] | Development of a hybrid recommender system for Scientific Cultural Heritage (SCH) data using ML algorithms and semantic web technologies | SCH data collected from the Drâa-Tafilalet region, Morocco | Analyze content and user preferences to create personalized recommendations of relevant SCH object | Data dispersion; refining profiles with collaborative filtering; |
[124] | Design personalized visiting paths using Context-Aware Recommender Systems (CARSs) and a mathematical model to optimize the number of visited Points of Interest (POIs) within a given timeframe | Contextual information | Effective personalization and optimization of visiting paths for tourists | Integration of diverse contextual data; |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Gîrbacia, F. An Analysis of Research Trends for Using Artificial Intelligence in Cultural Heritage. Electronics 2024, 13, 3738. https://doi.org/10.3390/electronics13183738
Gîrbacia F. An Analysis of Research Trends for Using Artificial Intelligence in Cultural Heritage. Electronics. 2024; 13(18):3738. https://doi.org/10.3390/electronics13183738
Chicago/Turabian StyleGîrbacia, Florin. 2024. "An Analysis of Research Trends for Using Artificial Intelligence in Cultural Heritage" Electronics 13, no. 18: 3738. https://doi.org/10.3390/electronics13183738
APA StyleGîrbacia, F. (2024). An Analysis of Research Trends for Using Artificial Intelligence in Cultural Heritage. Electronics, 13(18), 3738. https://doi.org/10.3390/electronics13183738