Ontology-Based Semantic Conceptualisation of Historical Built Heritage to Generate Parametric Structured Models from Point Clouds

Abstract

Nowadays, cultural and historical built heritage can be more effectively preserved, valorised and documented using advanced geospatial technologies. In such a context, there is a major issue concerning the automation of the process and the extraction of useful information from a huge amount of spatial information acquired by means of advanced survey techniques (i.e., highly detailed LiDAR point clouds). In particular, in the case of historical built heritage (HBH) there are very few effective efforts. Therefore, in this paper, the focus is on establishing the connections between semantic and geometrical information in order to generate a parametric, structured model from point clouds using ontology as an effective approach for the formal conceptualisation of application domains. Hence, in this paper, an ontological schema is proposed to structure HBH representations, starting with international standards, vocabularies, and ontologies (CityGML-Geography Markup Language, International Committee for Documentation conceptual reference model (CIDOC-CRM), Industry Foundation Classes (IFC), Getty Art and Architecture Thesaurus (AAT), as well as reasoning about morphology of historical centres by analysis of real case studies) to represent the built and architecture domain. The validation of such schema is carried out by means of its use to guide the segmentation of a LiDAR point cloud from a castle, which is later used to generate parametric geometries to be used in a historical building information model (HBIM).

1. Introduction

In the last decades, the necessity to document and store information related to cultural heritage (CH) and historical built heritage (HBH) has led researchers and experts to focus on the techniques and methods to more effectively represent historical buildings, geometric features and knowledge, spatially. Most recent methods and technologies in this respect adapt building information models (BIM), born to assist in building design and construction for the representation and management of heritage information, known as historical BIM (HBIM) [1,2]. However, the geometry usually stored in BIMs is parametrically modelled, while a large amount of the explicit geometric information (i.e., stored by means of the values of coordinates of single points) can be available for historical buildings from advanced survey techniques. The automatic interpretation of such information would be an advantage in the conversion of point clouds to BIM. In addition, the use of an effective ontology to define proper semantics for historical building objects already in this early modelling phase would greatly support the following management and interpretation of the building knowledge.

The process and the activities needed to preserve, valorise, restore, and document CH and HBH within spatial and temporal dimensions involve many communities, use cases and actors dealing with various data acquisition and processing phases. For this reason, a common language to semantically represent HBH is required.

An ontology or a complete conceptualisation that is able to semantically formalise historical buildings and HBH does not exist. In the last decades, many standards, vocabularies, and some ontologies for representing the built and architecture domain have been developed and adopted internationally (Section 2). However, none of these can spatially define entirely architectural elements of historical buildings and ancient structures, and their relations with the urban context and the historical centre in which they are located.

Considering the most widespread and accepted definition and understanding of the term ontology [3,4,5], and the significant role of ontologies with respect to knowledge engineering [6] and artificial intelligence, an ontological structure would be an effective solution for the formal conceptualisation of the HBH domain [7]. Hence, in this paper we propose an ontological structure useful from the early modelling phase of point cloud interpretation and segmentation to the later phase of management of the stored information. Such a workflow allows the connection between semantic and geometrical/topological information to generate structured models of built heritage.

Section 2 presents the state of the art on the relevant topics involved. Section 3 is dedicated to the methodology workflow, with a specific choice of archetypal case studies for HBH types included in castles and towers types (Section 3.1). Section 3.2 is aimed at analysing what the concepts are and identifying characters of such fortified structures by incorporating definitions and characteristics, and classifications and hierarchies of components according to standards and other published sources, as well as identifying ontology classes (objects) definitions (Section 3.3). Section 4 is devoted to the ontology development, and subsequently, the validation through semantic segmentation is presented starting from point clouds models (Section 5); the results are discussed, highlighting the connections with the ontological scheme.

2. Literature Review

In order to conceptualise and represent building components, several well-known standards have been developed during the past decades. This includes 3D city models (considering standards such as CityGML by the Open Geospatial Consortium) and Building Information Models (BIM), for which the reference standards are the Industry Foundation Classes (IFC) developed by buildingSMART. The standards consider a higher level of representation, especially concerning interior parts of buildings, structures, component materials, etc. However, starting from the introduction of the use of BIM technology for Heritage (HBIM) [8], it has become evident that one of the most problematic aspects of its application is the criticality of making the parametric approach compatible with the uniqueness of historical architecture. This is due to the geometric complexity of the artefacts, degradation and material characterisations: they are different from contemporary buildings, which count on a high level of industrialisation and, consequently, on higher repeatability of components and more simple shapes. Reverse engineering is usually necessary to suitably model historical buildings’ components.

This strategy and many consequent developments have been adopted to facilitate and better manage the maintenance of existing buildings [9,10], for conservative and restoration requirements [11,12] and also for the structural analysis and evaluation of historical architecture [13,14,15]. Precisely for these reasons, several efforts are directed to the database design with a particular focus on non-geometric information correlated to HBIM that is repeatable on different examples of heritage [16], with the harmonisation of vocabularies and repositories [17].

In the framework mentioned above, it is possible to collocate some already-made studies concerning standard classification in the built historical heritage domain; HBIM is linked to semantic-web studies and the ontology definition for historical centres and buildings. A starting point towards the creation of an ontology for small historical centres individuating useful classifications, descriptions, and concepts able to formalise the selected domain has been presented in [18]. Moreover, the development of identifying a path for historical fortified buildings and structures’ semantic definitions underlining the multitude of use cases concerning HBH and historical urban architecture is proposed 7.

Yang et al. [19] reported the worldwide diffusion of integrated approaches that combine BIM (or HBIM) techniques with information management of semantic knowledge. Lopez et al., (2018) [20] showed a significant strategy for our present work: they combined HBIM models and the Getty Art and Architecture thesaurus (AAT) (a vocabulary for architectural elements’ definitions, explained below) in SPARQL (an RDF query language), adding ID semantic information in the HBIM model parameters. This work presented a semantically enriched HBIM library for the elements of a castle.

Other studies [21,22] focused on the semantic enrichment of HBIM models (designed from a 3D metric survey) for the interpretation, management and querying of historical buildings organising knowledge using ontologies and databases to store hierarchical information and properties.

As ref. [23] attempted to define IFC in an ontology language (OWL), they connected IFC and RDF, creating a new structure which resulted in an ifcOWL ontology. Very recently, some authors [24] worked to define an ontology-based database providing a vocabulary to query an HBIM model of vaults, and ref. [25] studied an integration workflow connecting HBIM modelling and ontological knowledge.

Other research works attempt to use different standards and vocabularies from the geographic information domain for historical heritage representation. For instance, CityGML [26] is an international standard data model published by the Open Geospatial Consortium (OGC) in 2008 (developed since 2002) to represent multiscale 3D information about entities of cities. It is an open data model for the storage and exchange of 3D city models.

Concerning the architectural heritage, the IFC (Industry Foundation Classes) in the BIM (Building Information Model) environment for parametric modelling must be considered [27].

Regarding the documentation of CH, it is possible to mention two primary standards: the International Committee for Documentation (CIDOC)-CRM (CIDOC conceptual reference model) core ontology [28] and the Getty AAT (Art and Architecture Thesaurus). The CIDOC-CRM was developed by the International Committee for Documentation (CIDOC) of the International Council of Monuments (ICOM) and is the core ontology for managing cultural heritage information, now standard ISO 21127 [29]. This standard is aimed at enabling the exchange and integration of information between heterogeneous data sources. It was initially developed for the representation of the knowledge of museum objects.

The Getty vocabularies [30] propose terms connected to cultural heritage, as premiere references to categorise works of art, architecture, material culture, the names of artists or architects, as well as the geographic categories. One of these is the Art and Architecture Thesaurus (AAT) to describe works of art and architecture. Terms in the AAT may be used to describe art, architecture, decorative arts, material culture, and archival materials. The target audience includes museums, libraries, visual resource collections, archives, conservation projects, cataloguing projects, and bibliographic projects. Recently, the Getty vocabularies also developed a SPARQL version of the thesaurus available on-line [31].

In brief, despite many studies, there is no existing ontology or standard that provides a complete semantic formalisation of HBH and their elements. Nevertheless, there is a wealth of knowledge in terms of core ontologies and legislation documents that could serve as a basis for developing a more adapted domain ontology for the exploration of historical buildings. In this framework, it is possible to mention various studies focused on combining 3D modelling of HBH, precisely HBIM, and knowledge-based or ontological approaches, reusing existing knowledge and standards in the domain of geographic information and built heritage representation. Such an ontology can, subsequently, be used to bridge semantic and geometrical/topological information acquired with advanced technologies (e.g., LiDAR point clouds) to generate parametric structured models of built heritage for their effective documentation, preservation, valorisation, and restoration.

3. Proposed Ontology-Based Method for the Generation of Parametric Structured Models for Historical Built Heritage

Here we propose a workflow for the conceptualisation and 3D modelling of selected domains of HBH in particular castles and fortified structures in historical centres. The proposed workflow is composed of several steps that allow bridging between an ontological representation of the historical sites with their effective 3D representation using a detailed LiDAR data set. The present study adapts the following reproducible workflow (illustrated in Figure 1):

• The selection of castles and fortified structures as archetypal models and definitions, and the identification of geometrical characterisation (morphology) of building elements starting from case studies (Section 3.1).
• A survey and analysis of existing descriptions referred to as HBH in vocabularies and standards, to spatially represent them (Section 3.2).
• Reasoning about morphology from case studies analyses (Section 3.3).
• Ontology development (Section 4).
• Validation of the developed ontology by means of semantic segmentation of a case study point cloud by detecting building parts based on topological relations and conceptualisations defined in the ontology (Section 5).

This methodological approach follows three different levels, presented in the next sections:

• First level: concepts and relations derived from ontologies and standards.
• Second level: domain knowledge definition from instances (case studies).
• Third level: geometry definition and segmentation (through the 3D model design).

3.1. Identifying Specific Morphology through Archetypal Model

To identify the most representative morphological characters of castles and towers (from the medieval period), we have resorted to the constructive solutions of designers of the past who intended to convey universal or archetypal values of fortified structures with their works.

The first example is the Castellina fortress in Norcia (Figure 2) [32], commissioned by Pope Julius III and designed in 1554 by Jacopo Barozzi da Vignola, an architect and a relevant theorist and treaties writer of architecture in the Italian Renaissance panorama. The fortified structure blends the Renaissance building’s stylistic features with the fortress’s formal and constructive characteristics typical of the medieval period; the building is a square plan with four mighty corner towers and scarped masonry.

The second essential reference for this study is located in Turin, the Rocca (Figure 3), together with the connected Borgo Medievale, and was designed by Alfredo d’Andrade to symbolise the medieval Italian styles [33]. The Rocca, the innermost keep of a reconstructed hamlet, has the purpose of revisiting some real medieval architectures spread in two Italian regions, Piedmont and Valle d’Aosta. On the occasion of a crucial universal exhibition, the architect aimed to present the characteristics of native architecture to the attendants.

Starting from these examples that intrinsically contain the morphological aspects of construction practices of previous centuries for fortified buildings, we can identify the following distinctive characteristics:

• impressive masonries; buttresses masonries; sloping scarp walls;
• an angular tower (squared, polygonal and circular);
• small vertical openings and windows;
• a watchtower (in Italian “garitta” which is a sentry box);
• a merlon (in Italian “merli”, and they are at the top of the walls);
• the presence of a bulwark or moat.

3.2. Study and Analysis of Definitions of HBH to Identify Their Morphology

This step of the methodology aims at collecting elements and concepts related to fortified structures and castles’ architectural characters. It has been developed to compare conceptualisations, hierarchies, and descriptions from existing vocabularies and standards. This analysis allowed us to point out the most relevant elements and notions about castles and towers.

3.2.1. Concepts Related to Castles and Fortified Structures: The Getty AAT, CityGML and IFC as Standards

The present subsection compares concepts and definitions related to castles and fortified structures in the Getty AAT vocabulary, the CityGML and IFC standards.

Table 1. List of characters or elements concerning fortified structures, inferable by standard concepts correspondences.

Concepts Related to Castles and Fortified Structures	Concepts from Standards	Definitions
fortified structures, castles	Getty AAT: fortification	General term for any works made to oppose a small number of troops against greater numbers.
	Getty AAT: fortified settlements	Settlements of any kind with defensive structures such as moats, enclosures, or ramparts.
	Getty AAT: counterscarps	Refers to the outer slopes of encircling defensive ditches; for the inner slopes, use “scarps”.
	Getty AAT: scarps (fortification elements)	Steep slopes immediately in front of and below a fortification; for the outer slopes of encircling ditches of fortifications, use “counterscarps”.
	Getty AAT: fortification elements	Architectural elements used to fortify structures.
	Getty AAT: fortification openings	Openings located in fortresses or fortified structures.
building	CityGML-CityObject→Site→AbstractBuilding→Building	Allows the representation of thematic and spatial aspects of buildings, building parts and installations in four levels of detail: LOD1 to LOD4.
	CityGML-Building→Building parts	Building part is a subdivision of a building that is homogeneously related to its physical, functional or temporal aspects, and may be considered as a building.
	IfcSite→IfcBuilding	A building represents a structure that provides shelter for its occupants or contents and stands in one place.
	IfcSite→IfcBuilding→IfcBuildingElements	The building element comprises all elements that are primarily part of the construction of a building
towers	Getty AAT: antiaircraft towers	Large antiaircraft gun blockhouses used by the Luftwaffe in World War II to protect against Allied overflights, serve as air-raid shelters, and coordinate air defence. They had concrete walls up to 3.5 m thick and were considered invulnerable.
	Getty AAT: defensive towers	Towers used specifically for defence purposes.
	Getty AAT: watchtowers	Any built structure used as a lookout station that affords observation of a surrounding area.
wall	IfcBuildingElement→ifcWall	Vertical construction usually in masonry or in concrete which bounds or subdivides a construction works and fulfils a load bearing or retaining function. Definition from buildingSMART: The wall represents a vertical construction that bounds or subdivides spaces.
	CityGML CityObject→BoudarySurface→SurfaceWall	The embrasure surface on the one side of the door that belongs to the InteriorWallSurface and on the other side to the WallSurface.
	Getty AAT: defensive wall	Use broadly for all sorts of walls constructed to resist attack.
	Getty AAT: bulwarks	The strong outer works of a fortification, which are usually low enough to permit defensive fire.
moat	Getty AAT: moat	Refers to deep, wide defensive ditches surrounding towns, castles, or houses and are usually filled with water.
sentry box	Getty AAT: sentry box	Huts or boxes to shelter a sentry at his post.
merlons	Getty AAT: merlons	The solid portions in a battlement, which alternate with the embrasures.

shows the main entities.

3.2.2. Concepts and Definitions from the Italian Castles Institute to Identify Fortified Structures Morphology

The concepts of fortified structures extracted from standards or vocabularies are rather general; curiously, Getty ATT provides both comprehensive definitions of defensive towers and a specific one related to antiaircraft built structures used to provide shelter from aerial bombings during the Second World War.

Although it is essential to consider the era of construction of fortified structures, this study aims at identifying recurring features of historical fortifications to recognise morphological characteristics that connect the external shape with the geometric nature of the model (e.g., in HBIM), and the semantic values of objects.

Towers and castles are buildings intended for defence. They characterised the construction methods and systems directly connected to the ways of “making war”. For this reason, the disciplines of the history of architecture and the history of urban planning have been very interested in the evolution of fortified structures over the different centuries until the modern era. Due to the scarcity of definitions and information in the standards, we looked for other meanings and conceptualisations provided by scientific communities, such as medieval historians of architecture. An association of historians called the Italian Castelli Institute [34] provided a useful castle nomenclature dictionary (

Table 2. Useful terms and definitions to define castles and their elements.

Concepts	Description	Concepts from Definitions
castle	A fortified architectural or urban complex designed to defend a noble’s residence or in any case of a recognised authority; at their origins, in castles, the residential function joins the military one. Over time, the residential function prevailed; according to the epochs, we have distinguished it in feudal castle, noble castle, castle of the princes, castle palace, and castle villa. There are numerous derivatives but are, essentially, of the same meaning: castella, castelletta, castelluccio, castelletto, castellino, castellucchio, castellazzo, castellina and, the derogatory castellaccio.	Is-a: fortified structure, urban fortified complex Function: residential or military Purpose: defend a noble’s residence Subclasses: feudal castle, noble castle, castle of the princes, castle palace, castle villa
tower	Fortification with a millenary tradition, it was built as a lookout and signalling function, or as a warrior’s home, often as an integral element of a castle to strengthen and defend the walls or to defend the entrance. It can have a quadrilateral, circular, polygonal, semi-circular, elongated semi-circular plan, with parallel vertical walls open on the back. They were rarely built with a spur-shaped or almond-shaped spur. Defensive weapons of all kinds were installed there, from ballista to catapult; an effective swooping defence was practised from the top of the tower, exploiting the walls’ height, and the crowning was almost always equipped with arrows, crossbowmen or loopholes. A particular type, nearly exclusively Italian, is the urban tower, typical of medieval cities.	Is-a: fortification, fortified structure Function: lookout and signalling, warrior’s home Purpose: to strengthen and defend the walls or to defend the entrance Plan: quadrilateral, circular, polygonal, semi-circular, elongated semi-circular Subclasses: urban tower
sentry-box	A “bertesca” protruding from the walls, used for sighting or combat from above, with sidearms. Then, a turret was placed at the salient corners of the fortifications. Finally, it was a place of shelter for sentries. (in Italian: Garitta or Garetta)	Is-a: brattice Purpose: sighting or combat from above/shelter for sentries Has-part: sidearms Is-part-of: fortification
merlon	Masonry expedient of the upper parts of the castle walls and towers, well known under the iconographic profile. It consisted of a symmetrical interruption of the wall, behind which the shooter sheltered to escape from the opponent’s reaction. It is called Guelfo or Ghibellino, depending on whether it ended in par or dovetail but the denomination has not always corresponded to the castellans’ faction. Other and different shapes were adopted according to the places: triple-toothed, flower-shaped, pyramid-shaped, semicircle (for which the set of battlements was also called lace, prominence). In the most ancient feudal castles, the blackbird was built flush with the wall’s outer surface; it was later carved out of the projecting walls to allow a more effective swooping defence. It was made also tapered, double sloping, with curbs or frames to prevent it slipping of arrows or lightning bolts, and with slits.	Is-a: masonry Located-in: upper part of the castle Function: provide shelter to the shooter to escape from the opponent’s reaction Consists-of: symmetrical interruption of the wall Has-Shape: triple-toothed, flower-shaped, pyramid-shaped, semicircle, tapered, double sloping, dovetail
bulwarks	A defensive preparation adopted as a result of the use of artillery, to strengthen the meeting point of two curtains and, at the same time, to allow for more effective flanking defence. It was usually made up from a vast embankment covered with a wall, with a pentagonal plan (two faces, two sides and a groove), aligned with the bisector of the angle formed by two curtains adjacent and angled.	Is-a: defensive arrangement Purpose: increase the defensive possibilities Shape: pentagonal
moat	A canal dug around a castle or fortification, between the scarp and the counter scarp, to increase the defensive possibilities. It is clear how this was possible for fortifications on the ground level; in this case, a river’s course was diverted. It could have been an open moat, visible, flooded or dry, floodable or water operated, and also a blind moat, covered with reeds or bundles, like a trap.	Is-a: canal Purpose: increase the defensive possibilities Subclasses: open moat, visible, or flooded, or dry, floodable or water operated, blind moat
courtyard	An architectural element that ended up constituting the most important and usually central nucleus of each era’s castle. Due to its configuration, it was distinguished in closed courtyards (quadrilateral or rectangular), open courtyards (U-shaped), two-sided perpendicular courtyards (L-shaped), and double open (H shaped).	Is-a: Building part Part-of: castle Subclasses: closed courtyard (quadrilateral or rectangular), open courtyard (U-shaped), two-sided perpendicular courtyard (L-shaped), double open (H shaped)

).

These notes as mentioned above are basic definitions for castles, and many of them contribute to understanding fortified structures’ morphology; moreover, we can see that there are many architectural elements or elements of the type of construction that characterise castles and towers (especially from the medieval period) that allow us to identify them clearly.

The structures’ architectural morphology enables the identification of geometric and topological rules for the model generation (some examples in Figure 4).

3.3. Reasoning about Morphology from Case Studies Analysis

We considered these two built heritage samples (the Castellina and the Rocca medioevale) and the related 3D models to link the definitions with the identification of the morphological aspects because, in the architects’ intentions, they both embody the archetypal image of the castle and towers.

In this section the recurring morphological elements have been analysed, listing concepts and geometric/topological features:

• sloping scarp walls: the scarp masonry and the portion of the vertical wall with which it is connected; they can be identified since they are two contiguous planar surfaces, which share a line that is the intersection between planes.
  The sloping scarp masonry can be noticed in the Castellina. In the XIX century construction, the scarp walls are missing since other typical features and elements of the fortress were enough to communicate castles’ general configuration.
• towers: as the definition states, a tower has a simple shape, with a square, circular, or polygonal plan. Then, the definition also includes more complex open shapes, but the most common towers are featured by a square or circular base.
  Morphology: if the towers are isolated, their geometric characterisation corresponds to a cylinder in the 3D space or a parallelepiped with a square base. If they are not separated, but placed at the corners of the castle, we must first examine the morphology of the court, since the organisation around a courtyard with angular towers is very typical.
  Another geometric feature is that the towers are higher than the fortified building or castle to which they are related. It can also be argued that the structures, for many centuries, were built with few floors above ground level, so castles and towers are higher than the average height of the buildings of medieval towns.
• courtyard: a courtyard is an open-air place, from the geometric point of view; the centroid of the volume of the entire building falls within an empty area and not within its projected polygon (Figure 5).
• windows and openings: in modern construction, starting from the nineteenth-century regulations, indoor environments’ healthiness, lighting and ventilation must be guaranteed. Therefore, the present construction rules provide for minimum surfaces of building openings. Simultaneously, the window openings’ surfaces in towers and castles cover a much lower percentage than the minimum characters envisaged today or in historic buildings not designated for defence.

4. Ontology Development

The present section is dedicated to defining an ontology structure following the conceptualisations and definitions carried out in the methodological workflow. The ontology has been designed in Protègè (an open-source ontology editor and framework for building intelligent systems) [35]. Here, different classes (concepts with definitions) with their relations (object properties), characteristics and values (data properties and datatypes) have been added. Moreover, thanks to the implemented tools in Protègè, it was possible to integrate and merge existing ontologies of IFC and CityGML.

To enrich the first level (Figure 1, concepts and relation), with information relevant to topology and other useful elements for generating the second level (the domain knowledge), we also considered conceptualisations from CIDOC CRM (the core ontology for cultural heritage, by the International Committee for Documentation), CIDOC CRMba (for the documentation of archaeological buildings) and CIDOC CRMgeo (for spatiotemporal properties of temporal entities and persistent items) ontologies.

Figure 6 shows the hierarchy classes with their properties. Various classes derive from different standards, ontologies and reasonings about HBH elements. A particular focus has been paid to AbstractBuilding and building classes (building element and building part) definitions from IFC and CityGML, defining meronymic (has-part) and hierarchical relations. Moreover, classes from the cultural heritage domain, such as the architectural element, castle and fortified structure have been added.

Every concept in the developed ontology has URI (Uniform Resource Identifier) and a description derived by standards analyses (above reported) or CIDOC-CRM ontologies documentation.

Table 3. CIDOC concepts definitions.

CIDOC Concepts	Definitions
Built Work	Man-made things such as freestanding buildings, components of buildings, and complexes of buildings.
Morphological Building Section	Man-made things that are considered functional units for the whole building (e.g., rooms, foundations, roof, and so forth).
Filled Morphological Building Section	Man-made things that are completely filled with matter. Instances of filled morphological building section are components of the functional units of a built work. These are elements like walls, floors and ceilings
Physical Man-Made Thing	This class comprises all persistent physical items that are purposely created by human activity. This class comprises man-made objects, such as a sword, and man-made features, such as rock art.
CRM Entity	This class comprises all things in the universe of discourse of the CIDOC conceptual reference model.
Person	This class comprises real persons who live or are assumed to have lived.
Place	This class comprises extents in space (particularly on the surface of the earth) in the pure sense of physics, independent from temporal phenomena and matter.
Phenomenal Place	This class comprises instances of place whose extent and position are defined by the spatial projection of the spatiotemporal extent of a real-world phenomenon that can be observed or measured.
Time Span	This class comprises abstract temporal extents, in the sense of Galilean physics, having a beginning, an end and duration.
Phenomenal Spacetime Volume	This class comprises the four-dimensional point sets (volumes) which material phenomena occupy in space–time.
Type	This class comprises concepts denoted by terms from thesauri and controlled vocabularies used to characterise and classify instances of CRM classes.

presents an example of definitions from CIDOC.

The ontologies’ properties have been implemented in Protègè by using object properties and data properties (Figure 7) They correspond to the relationships between concepts (entities/classes defined) and their attributes or values. Some of them are ad-hoc created for the domain ontology of historical built heritage; others are integrated and imported from existing ontologies (such as CIDOC-CRM). Figure 8 shows entities and relations in the WebVOWL application, and

Table 4. Some other object properties of the historical built heritage (HBH) ontology.

Class Domain	Object Property	Class Range
moat	borders with	counterscarp
moat	borders with	bulwark
castle	borders with	moat
castle	has constructive element	sloping scarp wall
castle	has constructive element	watchtower
castle	has constructive element	merlon
angular tower	has architectural element	small window
building	has-part	building element
courtyard	is-part-of	castle
wall	has-part	window
tower	has-part	wall
built work	located	place
person	has type	type
built work	has type	type
morphological building section	has type	type
built work	designed in	time span
built work	designed by	person
morphological building section	is connected through	morphological building section
morphological building section	is section of	built work
built work	has time span	time span
	has shape
…	…	…

and

Table 5. Excerpt of data properties and datatype of HBH ontology.

CLASSES	Data Property	Datatype	Can Be Populated with…
tower	function	literal	military/residential
castle	function	literal	military/residential
small window	architectural shape	literal	vertical
angular tower	shape	literal	quadrangular/circular
tower	typology	literal
castle	typology	literal
courtyard	typology	literal	inner
window	dimension	decimal
…	…	…

show the reported list of properties with their domain and range entities and datatypes.

One of the first final outputs of the presented methodology is formalising the selected domain of HBH concepts into the ontology scheme.

If the ontological structure is open and standards-based, then it could be reused, integrated and implemented based on new case studies and examples of historical built heritage and castles.

Moreover, for the third level of the methodology’s workflow, we present a partial example of the conceptual formalisation in the case study of Castellina of Norcia by using some indicative concepts and relations considered in the ontology development (Figure 8 and Figure 9).

5. Validation: The Use of the Ontology to Extract a Historical Site Component from LiDAR Data

This section explains the validation of the proposed methodology. For this purpose, we present a case study for the extraction of the components of a historical building through a semantic reasoning approach applied on a set of detailed 3D LiDAR point clouds (Figure 10).

5.1. LiDAR Data Acquisition

A complete UAV (unmanned aerial vehicle) survey has been carried out to obtain high-detailed point clouds for Norcia main square and Castellina [36,37]. Over the analysed areas, some low altitude UAV flights were performed using the Phantom 4 PRO and DJI Spark platforms in order to accurately describe the outer part of the castles (an average elevation flight height of 50 m was used). Complying with the consolidated trends in UAV image acquisition, the flights over the fortified structures were planned to acquire nadir and oblique images, to obtain a GSD (ground sampling distance) with a magnitude of less than 2.5 cm, which is suitable for an architectural scale (1:100–1:200).

A geo-reference system was established thanks to the integration of a principal network acquired with a GNSS system, and a vast amount of artificial and natural targets that were measured to obtain a reliable network of controls and checkpoints, useful to optimise and evaluate the bundle block adjustment of the photogrammetric process.

5.2. Semantic Segmentation of LiDAR Point Cloud for Object Extraction

For the purpose of the extraction of the components of a historical site from LiDAR point clouds presented in the previous section, we chose to use a machine-learning approach. These methods allow more automatic segmentation and extraction of information from the LiDAR data. With the machine learning-based solutions, we attempted to regroup the points belonging to each predefined class of objects in the proposed HBH ontology. This necessitates the training of a classifier on a data set as the first step.

For this purpose, a feature vector is designed to describe the properties of each point. The feature vector generally consists of quantitative values for each feature. For the task of extracting points belonging to our historical building from point clouds, semantic segmentation based on the feature vectors proposed in [38] is chosen to process the input point clouds presented in the previous section. In this work, among others, features such as

(i)

“directional height above” compares the height difference between a point and its neighbours in eight directions,

(ii)

“difference of normal” is based on a normal estimation which can better distinguish when vegetation and man-made objects are used.

Then, following semantic segmentation of the point cloud, they were used in a CAD-like segmentation. This approach decomposes historical buildings into their components based on their geometric properties. Each building component is identified as a geometric shape from point clouds. The normal estimation is commonly used to extract the geometric properties such as smoothness of surfaces from point clouds.

A region-growing algorithm is applied to detect the smoothness of surfaces through predefined criteria (e.g., the proximity, smoothness, and curvature) of growing surfaces by comparing the properties of adjacent points. The smoothness is evaluated by the angle between the normal vectors based on the normal estimation of point clouds. An appropriate smoothness threshold depending on point cloud quality (e.g., point density and noise) can mitigate cases with under-segmentation and over-segmentation results. For deciding the smoothness threshold automatically from point clouds, Otsu’s method [39,40] was chosen to help to mitigate the interference of edge points during the growing regions. We ran the region-growing segmentation algorithm continually on the segmentation results from the previous step to decrease under-segmentation results for the case that two planar surfaces connect with a gentle angle. Then RANSAC (RANdom SAmple Consensus) algorithm [41] is used to estimate the geometric parameters of primitives representing building components.

5.3. Extraction of Topological Relations between Geometric Components

The topological relations between the components are summarised as a 3*3 matrix of the dimensionally extended 9-intersection model (DE-9IM) [42] to represent topological relations. As the description presented in [43], the matrix is composed of the dimension operation on the interior, the boundaries of the two planar regions that represent building elements, and the intersection line between the two plane regions in 3D space. Based on the representation of DE-9IM for recording topological relations between planar regions, the semantic representation of the topological relations between the two planar regions is determined by analysing the relation between the intersection line and two planar regions, and the relations between the intersected parts composed of the intersection operation between planar regions and the intersection line. For example, the topological relations between the intersected parts could be “point-point” relations, “point-line segment” relations and “line segment-line segment” relations.

For describing the topological relations in a semantically meaningful way, the formalised representation of topological relations between the planar regions is described by the four-word description “T_p1-T_p2-T_p3-T_p4”. The first part T_p1 describes the overall topological relations between the two planar regions. It could be disjointed, met, and intersected. The second part T_p2 represents the relation between planar region A and the intersection line. It could be disjointed, met, and overlapped. The third part T_p3 represents the relation between planar region B and the intersection line. It could be disjointed, met, and overlapped as well. The fourth part T_p4 represents the relation between the intersected parts on the intersection line. It depends on the intersected parts created by the planar regions and the intersection line. When it is the relation between two points, it could be disjointed and equal. When it represents the relation between a line segment and a point, it could be disjointed, met and contained. When it represents the relation between two-line segments, it could be disjointed, met, overlapped, covered, covered by, contained, contained by, and equal. Finally, the topological relations between two planar regions in 3D space are formalised as semantic descriptions.

Based on the segmentation results of HBH buildings from point clouds, the geometric information of the building components is calculated from segments, and the topological relations between the building components can be identified as well. The geometric information of building components and the topological relations between components make it possible to extract the semantic information related to the building structure constructed from basic geometric and topological information.

5.4. Semantic Reasoning for the Recognition of Historical Building Components

For the extraction and recognition of historical building components from point clouds, there are several critical steps as follows.

• First, building a knowledge base is required to formalise the knowledge about building structure and its components. To this end, we make use of our proposed HBH ontology. In addition, for semantic reasoning purposes, semantic rules are defined based on the concepts, relations, properties, as well as the different constraints on relations and properties. The rules are formalised in a human-readable format.
• Second, for linking the segmentation results of buildings and the knowledge base, the segmented building components are translated into instances of concepts. The geometric information is translated into the properties of instances. The geometric and topological relations are translated into relations among instances. The information related to instances is viewed as fact for reasoning in the knowledge base.
• Finally, the reasoning process is carried out to recognise the impressive historical building structure based on the segmentation results of buildings from point clouds and prior knowledge of the building structure. Depending on the definition of the semantic rules formalising knowledge, the recognition of the studied complex building structure can benefit from the definition of the connection between basic building components (e.g., wall, roof) based on formalised topological relations.

Figure 11 and Figure 12 illustrate the results from semantic segmentation of two historical buildings and the subsequent reasoning steps.

6. Discussion on the Historical Buildings Components Extraction Results

The results of the semantic segmentation are in accordance with the architectural reading of the buildings. This description and reasoning about ontology classes and relations was also useful for detecting and selecting properties for the ontology design.

As shown in

Table 6. Description and reasoning about ontology classes, relations and properties.

Description and Reasoning about Ontology Classes, Relations and Properties				Geometrical/Spatial or Topological Constraints		Semantic Segmentation
Castle	Include →	walls	(typology) →	Scarp walls	Vertical and inclined surfaces share a line	Detected/achieved
Castle	Include →	courtyard	(typology) →	Inner courtyard	The centre of gravity of the projection of the buildings is outside of the polygon	Indirectly achieved
Castle	Include →	Tower	Include →	Walls (as above)/scarp walls	Vertical and inclined surfaces share a line	Detected/achieved
		Angular Tower	Typology →	Quadrangular/circular	Tower stop parallel buildings	Indirectly achieved
				sentry box (garitta)		Detected/achieved
		Walls	Include →	Openings/windows	Interruptions of the masonry surface which constitute a very small percentage of the entire masonry	Small openings detected

the vertical walls and scarp walls are identified by analysing the geometric relations between the segments and ground. After the topological detection, the vertical walls above the scarp walls are distinguished. Based on the knowledge that a tower includes a vertical and inclined surface, the group of walls, colourised as red, and the scarp walls are identified as the tower. The opening can be detected from the walls as well.

Further, Table 6 shows the ontological classes and relations detected in the morphology analysis used for the semantic segmentation of the point clouds of the case studies. The third column expresses whether the segmentation was able to detect the object classes (for example, in the first line we can assert that the semantic segmentation detected the scarp walls from the vertical walls. In the second line, we can say that if the semantic segmentation detected the buildings bodies, it indirectly detected the courtyard, which is the space located inside the building wings).

As can be observed in the previous paragraphs, the case studies’ point clouds representing urban scenes and buildings were segmented according to the terms, rules and topological relations defined in the ontology.

In this sense, the proposed ontology-based method for the generation of parametric structured models for historical built heritage is directly connected to the semantic segmentation, which is one of the validated part of the methodological solution.

The combination of these different approaches is, therefore, the main challenge that we want to address by proposing the described novel workflow.

7. Critical Conclusions and Future Work

Certainly, harmonising the rules for defining the geometric and semantic contents of the 3D models of built environments is one of the most relevant challenges facing the framework of spatial and urban information management. When these purposes assert themselves in complex cultural and historical heritage domains, which involves considering parts of knowledge that is reciprocally related, the issue becomes more complex and an interdisciplinary approach is highly recommended. The workflow experimented with and presented in this paper shows that the direction is valid for a particular historical heritage sample, such as castles and towers. Moreover, the methodology can be extended to many other morphological featured domains by experimenting with proper rules of semantic segmentation.

The designed ontology incorporates concepts and relations from different ontologies and vocabularies that express different interacting views in the historical built heritage (HBH) domain. Semantic information derived from the various case studies was also used to enrich the knowledge related to fortified structures. Since the field of study is vast, such an ontology could be further used to define relations among semantic information and spatial 3D models (Figure 13).

On the other hand, by adopting the ontology-based approach or defining high-level ontological schemes, specific domain knowledge reuse will increase. The short-term perspective that seems most suitable is critically reviewing the structuring of HBIM models, designed according to current canons, and implementing it with the new classes, entities and relations expressed by the outlined ontology (Figure 14).

Finally, another future development could regard the BIM-GIS (Geographic Information Science) harmonisation by checking the geometry, data formats and attribute integrations through the insertion of the 3D model in an ontology-based GIS environment [44].