An Object-Based Semantic Classification Method for High Resolution Remote Sensing Imagery Using Ontology

Abstract

Geographic Object-Based Image Analysis (GEOBIA) techniques have become increasingly popular in remote sensing. GEOBIA has been claimed to represent a paradigm shift in remote sensing interpretation. Still, GEOBIA—similar to other emerging paradigms—lacks formal expressions and objective modelling structures and in particular semantic classification methods using ontologies. This study has put forward an object-based semantic classification method for high resolution satellite imagery using an ontology that aims to fully exploit the advantages of ontology to GEOBIA. A three-step workflow has been introduced: ontology modelling, initial classification based on a data-driven machine learning method, and semantic classification based on knowledge-driven semantic rules. The classification part is based on data-driven machine learning, segmentation, feature selection, sample collection and an initial classification. Then, image objects are re-classified based on the ontological model whereby the semantic relations are expressed in the formal languages OWL and SWRL. The results show that the method with ontology—as compared to the decision tree classification without using the ontology—yielded minor statistical improvements in terms of accuracy for this particular image. However, this framework enhances existing GEOBIA methodologies: ontologies express and organize the whole structure of GEOBIA and allow establishing relations, particularly spatially explicit relations between objects as well as multi-scale/hierarchical relations.

1. Introduction

Geographic object-based image analysis (GEOBIA) is devoted to developing automated methods to partition remote sensing (RS) imagery into meaningful image objects, and assessing their characteristics through spatial, spectral, texture and temporal features, thus generating new geographic information in a GIS-ready format [1,2]. There has been great progress compared to traditional per-pixel image analysis. GEOBIA has the advantages of having a high degree of information utilization, strong anti-interference, a high degree of data integration, high classification precision, and less manual editing [3,4,5,6]. Over the last decade, advances in GEOBIA research have led to specific algorithms and software packages; peer-reviewed journal papers; six highly successful biennial international GEOBIA conferences; and a growing number of books and university theses [7,8,9]. A GEOBIA wiki is used to promote international exchange and development [9]. GEOBIA is a hot topic in RS and GIS [1,8] and has been widely applied in global environmental monitoring, agricultural development, natural resource management, and defence and security [10,11,12,13,14]. It has been recognized as a new paradigm in RS and GIS [15].

Ontology originated in Western philosophy and was then introduced into GIS [16]. The concept of domain knowledge is expressed in the form of machine-understandable rulesets and is utilised for semantic modelling, semantic interoperability, knowledge sharing and information retrieval services in the field of GIS [16,17,18]. Recently, researchers have begun to attach importance to the application of ontology in the field of remote sensing, especially in remote sensing image interpretation. Arvor et al. (2013) described how to utilise ontology experts’ knowledge to improve the automation of image processing and analysis the potential applications of GEOBIA, which can provide theoretical support for remote sensing data discovery, multi-source data integration, image interpretation, workflow management and knowledge sharing [19]. Jesús et al. (2013) built a framework for ocean image classification based on ontologies, which describes how to build ontology model for low and high level of features, classifiers and rule-based expert systems [20]. Dejrriri et al. (2012) presented GEOBIA and data mining techniques for non-planned city residents based on ontology [21]. Kohli et al. (2012) provided a comprehensive framework that includes all potentially relevant indicators that can be used for image-based slum identification [22]. Forestier et al. (2013) built a coastal zone ontology to extract coastal zones using background and semantic knowledge [23]. Kyzirakos et al. (2014) provided wildfire monitoring services by combining satellite images and geospatial data with ontologies [24]. Belgiu et al. (2014a) presented an ontology-based classification method for extracting types of buildings where airborne laser scanning data are employed and obtained effective recognition results [25]. Belgiu et al. (2014b) provided a formal expression tool to express object-based image analysis technology through ontologies [26]. Cui (2013) presented a GEOBIA method based on geo-ontology and relative elevation [27]. Luo (2016) developed an ontology-based framework that was used to extract land cover information while interpreting HRS remote sensing images at the regional level [28]. Durand et al. (2007) proposed a recognition method based on an ontology which has been developed by experts from the particular domain [29]. Bannour et al. (2011) presented an overview and an analysis of the use of semantic hierarchies and ontologies to provide a deeper image understanding and a better image annotation in order to furnish retrieval facilities to users [30]. Andres et al. (2012) demonstrate that expert knowledge explanation via ontologies can improve automation of satellite image exploitation [31]. All these studies focus either on a single thematic aspect based on expert knowledge or on a specific geographic entity. However, existing studies do not provide comprehensive and transferable frameworks for objective modelling in GEOBIA. None of the existing methods allows for a general ontology driven semantic classification method. Therefore, this study develops an object-based semantic classification methodology for high resolution remote sensing imagery using ontology that enables a common understanding of the GEOBIA framework structure for human operators and for software agents. This methodology shall enable reuse and transferability of a general GEOBIA ontology while making GEOBIA assumptions explicit and analysing the GEOBIA knowledge corpus.

2. Methodology

The workflow of the object-based semantic classification is organized as follows: in the ontology-model building step, land-cover models, image object features and classifiers are generated using the procedure described in Section 2.2 (Step 1, Figure 1). The result is a semantic network model. Subsequently, the remote sensing image is classified using a machine learning method and the initial classification result is imported into the semantic network model (Step 2, Figure 1), which is described in Section 2.3. In the last step, the initial classification result is reclassified and validated to get the final classification result based on the semantic rules (Step 3, Figure 1), which is described in Section 2.4. The semantic network model is the interactive file between the initial classification and the semantic classification.

2.1. Study Area and Data

The test site is located in Ruili City, in Yunnan Province, China. We utilised panchromatic (Pan) data from the Chinese ZY-3 satellite with 2.1 m resolution and multispectral (MS) ZY-3 data with 5.8 m resolution (blue, green, red and near-infrared bands), which were acquired in April 2013. The ZY-3 MS imagery was obtained and geometrically corrected to the Universal Transverse Mercator (UTM) projection and then re-sampled to 2.1 m to match the Pan image pixel size; it was then fused using the Pansharp fusion method within the PCI Geomatica software. Figure 2 shows the resulting fused image based on MS bands 4 (near-infrared), 3 (red) and 2 (green).

The part of the city selected for the study is characterised by classes identified as Field, Woodland, Grassland, Orchard, Bare land, Road, Building and Water. The eight land-covers are defined based on the Geographical Conditions Census project in China [32], which are described as follows: Field is often cultivated for planting crops, which includes cooked field, new developed field and grass crop rotation land. It is mainly for planting crops, and there are scattered fruit trees, mulberry trees or others. Woodland is covered by natural forest, secondary forest and plantation, which includes trees, bushes, bamboo, etc. Grassland is covered by herbaceous plants, which includes shrub grassland, pastures, sparse grassland, etc. Orchard is artificially cultivated for perennial woody and herbaceous crops. It is mainly used for collecting fruits, leaves, roots, stems, etc. It also includes various trees, bushes, tropical crops and fruit nursery, etc. Bare land is a variety of natural exposed surface (forest coverage is less than 10%). Road is covered by rail and trackless road surface, including railways, highways, urban roads and rural roads. Building includes contiguous building areas and individual buildings in urban and rural areas. Water includes all types of surface water.

2.2. Ontology Model for GEOBIA

2.2.1. Ontology Overview

As stated in the introduction section, ontology plays a central part in this methodology. It is used to reduce the semantic gap that exists between the image object domain and the human language centred class formulations of human operators [3,14,19]. The ontology serves as the lynchpin to combine image classification and knowledge formalization. Ontology models are generated for land-cover, image object features, and for classifiers. An ontology is a formal explicit description of concepts and includes: classes (sometimes called concepts), properties of each class/concept describing various features and attributes (slots, sometimes called roles or properties), and restrictions on slots (facets, sometimes called role restrictions). An ontology together with a set of individual instances of classes constitutes a knowledge base [33]. There are many ontology languages, such as Ontology Web Language (OWL), Extensible Markup Language (XML), Description Logic (DL), Resource Description Framework (RDF), Semantic Web Rule Language (SWRL), etc. Ontology building methods include enterprise modelling, skeleton, knowledge engineering, prototype evolution, and so on. There are several ontology building tools, such as ontoEdit, ontolingua, ontoSaurus, WebOnto, OilEd, Protégé, etc., and there are several ontology reasoning machines (Jess, Racer, FaCT++, Pellet, Jena, etc.).

In this study, the information for land-cover, object features and machine learning classifiers are expressed in OWL while the semantic rules are expressed in SWRL. The OWL is defined as a recommended standard of ontology language by W3C which is based on the description logic. The relationship of concept and various semantics are expressed by XML/RDF syntax. OWL can describe four kinds of data: class, property, axiom and individual [34]. SWRL is a rule description language which includes OWL-DL, OWL-Lite, RuleML. The knowledge is expressed in OWL by a highly abstract syntax and the combination of Hom-like gauge [35]. The knowledge engineering method and the Protégé software developed by Stanford University have been chosen to build the ontology model for GEOBIA.

Our knowledge engineering method consists of seven steps:

Step 1

Determine the domain and scope of the ontology.

The domain of the ontology is the representation of the whole GEOBIA framework, which includes the information on various features, land-covers and classifiers. We used the GEOBIA ontology to combine land-cover and features for image classification.

Step 2

Consider reusing existing ontologies.

Reusing existing ontologies may be a requirement if our system needs to interact with other applications that have already been committed to particular ontologies or controlled vocabularies [33]. There are libraries of reusable ontologies on the Web and in the literature. For example, we can use the ISO Metadata [36], OGC [37], SWEET [38], etc. For this study, we assumed that no relevant ontologies exist a priori and start developing the ontology from scratch.

Step 3

Enumerate important terms in the ontology.

We aimed to achieve a comprehensive list of terms, For example, important terms include different types of land-cover, such as PrimarilyVegetatedArea, PrimarilyNonVegetatedArea, and so on.

Step 4

Define the classes and the class hierarchy.

There are three main approaches in developing a class hierarchy: top-down, bottom-up, combination. The approach to take depends strongly on the domain [33]. The class hierarchy include land-covers, image object features, classifiers, and so on.

Step 5

Define the properties of classes.

The properties become slots attached to classes. A slot should be attached at the most general class that can have that property. For example, image object features should be attached to the respective land-cover.

Step 6

Define the facets of the slots.

Slots can have different facets describing the value type, the allowed values, the number of the values (cardinality), and other features of the values the slot can take. For example, the domain of various features is “Region”, the range is “double”.

Step 7

Create instances.

Defining an individual instance of a class requires: (1) choosing a class; (2) creating an individual instance of that class; and (3) filling in the slot values [33]. For example, all the segmentation objects are instances, which have their properties.

Step 8

Validation.

The FaCT++ reasoner is used to infer the relationship among all the individuals, it could test the correctness and validity of the ontology.

Following this eight-step process, we designed ontology models for GEOBIA, namely for land-cover, image object features, and classifiers. Then, the semantic network model is formed.

2.2.2. Ontology Model of the Land-Cover

The Land Cover Classification System (LCCS) includes various land cover classification schemes [39]. In this study, we designed an upper level of classes based on the official Chinese Geographical Conditions Census Project [32] and the upper level of LCCS.

The ontology model of the eight land-covers is created as follows.

(1)

A list of important terms, including Fields, Woodland, Grassland, Orchards, Bare land, Roads, Building and Water, was created.

(2)

Classes and class hierarchies were defined. Land cover was defined through the top–down method and was divided into PrimarilyVegetatedArea and PrimarilyNonVegetatedArea. PrimarilyVegetatedArea was divided into ArtificialCropVegetatedArea and NaturalGrowthVegetatedArea. PrimarilyNonVegetatedArea was divided into ArtificialNonVegetatedArea and NaturalNonVegetatedArea. ArtificialCropVegetatedArea is divided into Field and Orchard. NaturalGrowthVegetatedArea is divided into Woodland and Grassland. ArtificialNonVegetatedArea is divided into Building and Road. NaturalNonVegetatedArea is divided into Water and Bare land. The classes are shown in Figure 3. Detailed classes can be defined according to the actual situation.

2.2.3. Ontology Model of the Image Object Features

Feature selection is an important step of GEOBIA as there are thousands of potential features describing objects. Some of the major categories include: layer features (marked as LayerProperty), geometry features (marked as GeometryProperty), position features (marked as PositionProperty), texture features (marked as TextureProperty), class-related features (marked as ClassProperty), and thematic index (marked as ThematicProperty). The ontology model makes use of the feature concepts used in the eCognition software to develop a general upper level ontology [40]. The image object features are defined through the top–down method and are divided into six categories: LayerProperty, GeometryProperty, PositionProperty, TextureProperty, ClassProperty, and ThematicProperty. Each feature category can be subdivided further. For instance, the TextureProperty is divided into ToParentShapeTexture and Haralick. The Haralick (which stands for Haralick’s texture GLCM parameters) is divided into GLCMHom, GLCMContrast and GLCMEntropy as illustrated in Figure 4.

2.2.4. Ontology Model of the Classifiers

Ontology is employed to express two typical algorithms, namely, decision tree and semantic rules.

(1)  Ontology model of the decision tree classifier

The ontology model of the decision tree classifier is based on C4.5 algorithm, which is specified by a set of nodes and leaves where the nodes represent Boolean conditions on features and the leaves represent land-cover classes. It is defined as follows.

(a)

A list of important terms, including DecisionTree, Node and Leaf, was created.

(b)

The slots were defined, which includes relations such as GreaterThan or LessThanOrEqual.

(c)

The lists of instances of decision tree, such as Node1, Node2, etc., were created. The nodes are associated to features and are also linked to two nodes with object properties called GreaterThan and LessThanOrEqual.

The ontology model of the decision tree classifier is shown in Figure 5.

(2)  Ontology model of the semantic rules

The process of modelling semantic rules includes building mark rules and decision rules. Building mark rules is based on a semantic concept, and the process is from low-level features to semantic concepts. Decision rules are obtained based on mark rules and a priori knowledge; the process is from advanced features to the identification of land-covers. The ontology model of mark rules and decision rules are shown as follows:

(a)  Ontology model of the mark rules

The objects are modelled from different semantic aspects, and, according to the common sense knowledge, it is divided into: Strip and Planar from the Morphology; Regular and Irregular from the Shape; Smooth and Rough from the Texture; Light and Dark from the Brightness; High, Medium and Low from the Height; and Adjacent, Disjoint and Containing from the Position relationship. The ontological model of the mark rules is created as follows.

a)

A list of important terms, including Morphology, Shape, Texture, Brightness, Height, Position, etc., was created.

b)

Class hierarchies were defined. Morphology was divided into Strip and Planar; Shape was divided into Regular and Irregular; Texture was divided into Smooth and Rough; Brightness was divided into Light and Dark; Height was divided into High, Medium and Low; and Position was divided into Adjacent, Disjoint and Containing.

The ontology model of the mark rules is shown in Figure 6.

The mark rules are expressed in SWRL, and the semantic relationships between the object features and the classes are built. For example, the Brightness type is expressed in SWRL as follows:

• Mean (?x, ?y), greaterThanOrEqual (?y, 0.38) -> Light (?x);
• Mean (?x, ?y), lessThan (?y, 0.38) -> Dark (?x).

This means the Mean feature of an object ≥0.38 denotes Light, whereas that <0.38 denotes Dark. C(?x), X is an individual of C, P(? X? Y) represents attributes, and x and y are variables.

(b)  Ontology model of the decision rules

The decision rules for eight types of land-covers are acquired from literatures, priori knowledge and project technical regulations. In general, the decision rules formalized using OWL are as follows:

• Field ≡ Regular ∩ Planar ∩ Smooth ∩ Dark ∩ Low ∩ adjacentToRoad.
• Woodland ≡ Irregular ∩ Planar ∩ Rough ∩ Dark ∩ High ∩ adjacentToField.
• Orchard ≡ Regular ∩ Planar ∩ Smooth ∩ Dark ∩ Medium ∩ adjacentToField.
• Grassland ≡ Irregular ∩ Planar ∩ Smooth ∩ Dark ∩ Low∩adjacentToBuilding.
• Building ≡ Regular ∩ Planar ∩ Rough ∩ Light ∩ High ∩ adjacentToRoad.
• Road ≡ Regular ∩ Strip ∩ Smooth ∩ Light ∩ Low ∩ adjacentToBuilding.
• Bare land ≡ Irregular ∩ Planar ∩ Rough ∩ Light ∩ Low.
• Water ≡ Irregular ∩ Planar ∩ Smooth ∩ Dark ∩ Low.

The decision rules are expressed in SWRL, and the semantic relationships between the mark rules and the classes are built. For example, the Field is expressed in SWRL as follows:

Regular (?x), Planar (?x), Smooth (?x), Dark (?x), Low (?x), adjacentToRoad (?x) -> Field (?x).

This means an image object with Regular, Planar, Smooth, Dark, Low and adjacentToRoad features is a Field.

Other classifiers such as Support Vector Machines (SVM), or Random Forest could be expressed in OWL or SWRL. Later on, the ontology model of the semantic rules can be extended and supplemented to realize the semantic understanding of various land-covers.

2.2.5. Semantic Network Model

The entire semantic network model is formed through the construction of the land-covers, image object features and classifiers using ontology. It is shown in Figure 7.

The semantic network model is a type of directed network graph that expresses knowledge through the concept and its semantic relations. It has the following advantages. Firstly, the concepts, features, and relationships of geographical entities are expressed explicitly, which could reduce the semantic gap between low-level features and high-level semantics. Second, it can be traced back to the parent object, child objects and neighbourhood objects through their relationships. Third, it is easy to express semantic relations using a computer operable formal language [41].

2.3. Initial Classification Based on Data-Driven Machine Learning

The process includes segmentation, feature selection, sample collection and initial classification. The software FeatureStation developed by the Chinese Academy of Surveying and Mapping is chosen to be the image segmentation and classification tool since it has from its onset on centred around segmentation and decision tree classification. The Protégé plugin developed by Jesús [20] is chosen as the semantic classification tool and for the transformation.

2.3.1. Image Segmentation

The objective of image segmentation is to keep the heterogeneity within objects as small as possible, at the same time preserving the integrity of the object. The fusion image is segmented using the G-FNEA method which is based on graph theory and fractal net evolution approach (FNEA) within the FeatureStation software. The method could get high efficiency and maintain good feature boundaries [42].

There are three parameters in the G-FNEA method: T (scale parameter), w_colour (weight factor for colour heterogeneity), and w_compt (weight factor for compactness heterogeneity). A high T value indicates fewer, larger objects than a low T value. The colour heterogeneity w_colour describes the spectral information, which is used to indicate the degree of similarity between two adjacent objects. The higher the w_colour value, the greater influence colour has on the segmentation process. The w_compt value reflects the degree of clustering of the pixels within a region: the lower the value, the more compact the pixels are within the region. It should be noted that the scale parameter is considered to be the most important factor for classification as it controls the relative size of the image objects and has a direct effect on the overall classification accuracy.

There are some methods on automatic determination of appropriate segmentation parameters, such as Estimation of Scale Parameters (ESP) [43], Optimised image segmentation [44], SPT (Segmentation Parameter Tuner) [45], Plateau Objective Function [46]. In this study, the selection of image segmentation parameters is based on an iterative trial-and-error approach that is often utilized in object-based classification [6,10]. The best segmentation results were achieved with the following parameters: T = 100, w_colour = 0.8, and w_compt = 0.3.

2.3.2. Feature Selection

The selection of appropriate object features can be based on a priori knowledge, or can make use of feature-selection algorithms (such as Random Forest [47]). In this study, we make use of a priori knowledge to guide the initial selection of object features, and thus keep to the following four rules: (1) the most important features of an object are the spectral characteristics, which are independent of test area and segmentation scale; (2) the ratio of bands is closely related to vegetation and non-vegetation; (3) the effect of the shape feature, which is used to reduce the image classification error rate, is small; therefore, it becomes effective when the segmentation scale reaches a certain level, that the objects are consistent with the real surface features; and (4) the auxiliary data (DEM, OpenStreetMap, etc.) is dependent on the scale; the smaller the scale, the more important the auxiliary data.

Based on the above four rules, twenty-nine features (e.g., ratio, mean, Normalized Difference Water Index, Normalized Difference Vegetation Index, homogeneity, and brightness) are selected and stored in Shapefile format, and then converted to OWL format. The features of an object in OWL is shown in Figure 8.

2.3.3. Initial Classification

The C4.5 decision tree method is used for the construction of a decision rule, which includes a generation stage and a pruning stage (Figure 9).

Stage 1: The generation of a decision tree

(1)

The training samples are ordered in accordance with the “class, features of sample one, features of sample two, etc.” The training and testing samples are selected by visual image interpretation with their selection being controlled by the requirement for precision and representativeness, and by their statistical properties.

(2)

The training samples are divided. The information gain and information gain rate of all the features of training samples are calculated. The feature is taken as the test attribute, whose information gain rate is the biggest and its information gain is not lower than the mean of all the features, and the feature is taken as a node and leads to a branch. In this circulation way, all the training samples are divided.

(3)

The generation of decision tree. If all the training samples of the current node belongs to a class, the class is marked as a leaf node and marked for the specify feature. It runs in the same way; at last, it forms a decision tree until all the data of a subset are recorded in the main feature and their feature value are the same, or there is no feature to divide again.

Stage 2: The pruning of decision tree.

The possible error probability of sub-node not leaf-node is calculated, the weights of all the nodes are assessed. The subtree is kept if the error rate causes by cutting off the node is high, otherwise, the subtree is cut off. At last, the decision tree with the least expected error rate is the final decision tree as shown in Figure 10. The decision tree is expressed in OWL as illustrated in Figure 11.

The above decision rule is imported into the semantic network model, all objects are classified using the decision rule, and the initial classification result is expressed in OWL file format.

2.4. Semantic Classification Based on Knowledge-Driven Semantic Rules

On the basis of the initial classification, each object is reclassified and validated by semantic rules in SWRL to obtain the semantic information.

2.4.1. Semantic Rules Building

The mark rules and decision rules of the eight classes of the test site are expressed in SWRL according to the ontology model of the above mark rules and decision rules.

(1)  Mark rules are shown as follows:

• RectFit (?x, ?y), greaterThanOrEqual (?y, 0.5) -> Regular (?x);
• RectFit (?x, ?y), lessThan (?y, 0.5) -> Irregular (?x);
• LengthWidthRatio (?x, ?y), greaterThanOrEqual(?y, 1) -> Strip (?x);
• LengthWidthRatio (?x, ?y), lessThan (?y, 1) -> Planar (?x);
• Homo (?x, ?y), greaterThanOrEqual (?y, 0.05) -> Smooth (?x);
• Homo (?x, ?y), lessThan (?y, 0.05) -> Rough(?x);
• Mean (?x, ?y), greaterThanOrEqual (?y, 0.38) -> Light (?x);
• Mean (?x, ?y), lessThan (?y, 0.38) -> Dark (?x);
• MeanDEM (?x, ?y), greaterThanOrEqual (?y, 0.6) -> High (?x);
• MeanDEM (?x, ?y), lessThan (?y, 0.2) -> Low (?x); and
• MeanDEM (?x, ?y), greaterThanOrEqual (?y, 0.2), lessThan (?y, 0.6) -> Medium (?x).

This means RectFit of an object >0.5 denotes Regular shape, where <0.5 denotes Irregular shape. The thresholds are obtained by an iterative trial-and-error approach.

(2)  Decision rules are shown by the following:

• Regular (?x), Planar (?x), Smooth (?x), Dark (?x), Low (?x), adjacentToRoad (?x) -> Field (?x);
• Irregular (?x), Planar (?x), Rough (?x), Dark (?x), High (?x), adjacentToField (?x)-> Woodland (?x);
• Regular (?x), Planar (?x), Smooth (?x), Dark (?x), Medium (?x), adjacentToField (?x) -> Orchard (?x);
• Irregular (?x), Planar (?x), Smooth (?x), Dark (?x), Low (?x), adjacentToBuilding (?x) -> Grassland (?x);
• Regular (?x), Planar (?x), Rough (?x), Light (?x), High (?x), adjacentToRoad (?x)-> Building (?x);
• Regular (?x), Strip (?x), Smooth (?x), Light (?x), Low (?x), adjacentToBuilding (?x) -> Road (?x);
• Irregular (?x), Planar (?x), Rough (?x), Light (?x), Low (?x) -> Bare land (?x); and
• Irregular (?x), Planar (?x), Smooth (?x), Dark (?x), Low (?x) -> Water (?x).

For example, an object with Regular, Planar, Smooth, Dark and Low features is a Field. C (? X), X is an individual of C, P (? X? Y) represents attributes, and x and y are variables.

2.4.2. Semantic Classification

The initial classification result is reclassified and validated to get the final classification result based on the semantic rules. The exported OWL objects are a way to preserve the semantics of the features the image objects exhibits.

3. Results and Discussion

3.1. Results

The description, picture, decision tree rules and decision rules of eight land-covers are shown in

Table 1. The description, picture, decision tree rules and decision rules of eight land-covers.

	Description	Picture	Decision Tree Rules	Ontology Rules	Decision Rules in SWRL Format
Field	Field is often cultivated for planting crops, which includes cooked field, new developed field and grass crop rotation land. It is mainly for planting crops, and there are scattered fruit trees, mulberry trees or others.		NDVI > 0.6 and RectangularFit > 0.62 and FractalDimension ≤ 0.37.	Field ≡ Regular ∩ Planar ∩ Smooth ∩ Dark ∩ Low ∩ adjacentToRoad.	Regular (?x), Planar (?x), Smooth(?x), Dark (?x),Low(?x), adjacentToRoad(?x) -> Field (?x)
Orchard	Orchard is artificially cultivated for perennial woody and herbaceous crops. It is mainly used for collecting fruits, leaves, roots, stems, etc. It also includes various trees, bushes, tropical crops and fruit nursery, etc.		NDVI > 0.6 and RectangularFit > 0.62 and FractalDimension > 0.37.	Orchard ≡ Regular ∩ Planar ∩ Smooth ∩ Dark ∩ Medium ∩ adjacentToField.	Regular (?x), Planar (?x), Smooth(?x), Dark (?x), Medium(?x), adjacentToField(?x) -> Orchard (?x)
Woodland	Woodland is covered of natural forest, secondary forest and plantation, which includes trees, bushes, bamboo, etc.		NDVI > 0.6 and RectangularFit ≤ 0.62 and Homogeneity > 0.71.	Woodland ≡ Irregular ∩ Planar ∩ Rough ∩ Dark ∩ High ∩ adjacentToField.	Irregular (?x), Planar (?x), Rough(?x), Dark (?x), High(?x), adjacentToField(?x) -> Woodland (?x)
Grassland	Grassland is covered of herbaceous plants, which includes shrub grassland, pastures, sparse grassland, etc.		NDVI > 0.6 and RectangularFit ≤ 0.62 and Homogeneity ≤ 0.71.	Grassland ≡ Irregular ∩ Planar ∩ Smooth ∩ Dark ∩ Low∩adjacentToBuilding.	Irregular (?x), Planar (?x), Smooth (?x), Dark (?x), Low(?x), adjacentToBuilding(?x) -> Grassland (?x)
Building	Building includes contiguous building areas and individual buildings in urban and rural areas.		NDVI ≤ 0.6 and MeanB1 > 0.38 and LengthWidthRatio ≤ 4.5.	Building ≡ Regular ∩ Planar ∩ Rough ∩ Light ∩ High ∩ adjacentToRoad.	Regular (?x), Planar (?x), Rough (?x), Light(?x), High(?x), adjacentToRoad (?x) -> Building(?x)
Road	Road is covered by rail and trackless road surface, including railways, highways, urban roads and rural roads.		NDVI ≤ 0.6 and MeanB1 > 0.38 and LengthWidthRatio > 4.5.	Road ≡ Regular ∩ Strip ∩ Smooth ∩ Light ∩ Low ∩ adjacentToBuilding.	Regular (?x), Strip (?x), Smooth (?x), Light(?x), Low(?x), adjacentToBuilding(?x) -> Road(?x)
Bare land	Bare land is a variety of natural exposed surface (forest coverage is less than 10%).		NDVI ≤ 0.6 and MeanB1 ≤ 0.38 and NDWI > 0.6.	Bare land ≡ Irregular ∩ Planar ∩ Rough ∩ Light ∩ Low.	Irregular (?x), Planar (?x), Rough (?x), Light (?x), Low (?x) -> Bare land(?x)
Water	Water includes all types of surface water.		NDVI ≤ 0.6 and MeanB1 ≤ 0.38 and NDWI ≤ 0.6.	Water ≡ Irregular ∩ Planar ∩ Smooth ∩ Dark ∩ Low.	Irregular (?x), Planar (?x), Smooth (?x), Dark (?x), Low (?x) -> Water(?x)

.

The initial classification result is expressed in OWL file format defined by the above steps of segmentation, feature selection, sample collection and initial classification. Figure 12 shows the classification result of “region208”, whereby “‘region208’is Water”. The expression of all the objects’ classification results is the same as “region208”.

On the basis of the initial classification, each object is reclassified and validated by semantic rules in SWRL to obtain the semantic information. Figure 13 shows a semantic classification result where “the semantic information of ‘region105’ is Regular, Planar, Smooth, Dark, Low, Field”, the expression of all the objects’ classification is the same as “region105”. Thus, the classified objects already exported in OWL format are help for retrieving the object features (Figure 14).

The semantic classification information in OWL format is transformed to Shapefile format, as shown in Figure 15a. A general object-based decision tree classification without ontology, which continues to use “image segmentation, feature extraction, image classification”, was investigated. The segmentation parameters and features are consistent in our method with ontology. The classification results are shown in Figure 15b.

A comprehensive accuracy assessment was carried out. A sample-based error matrix is created and used for performing accuracy assessment. In GEOBIA, a sample refers to an object. The error matrixes of the two methods for the test area are shown in Figure 16. The user’s accuracy, producer’s accuracy, overall accuracy and Kappa coefficient are shown in

Table 2. Land cover classification accuracy of the two methods.

Accuracy	Our Method with Ontology		Decision Tree Method without Ontology
	Production Accuracy (%)	User Accuracy (%)	Production Accuracy (%)	User Accuracy (%)
Field	88.14	86.67	85.00	77.27
Orchard	87.69	89.06	83.82	89.06
Woodland	85.11	88.89	91.11	95.35
Grassland	76.09	85.37	85.71	78.26
Building	85.71	75	82.86	80.56
Road	84.38	72.97	71.88	76.67
Bare land	90.38	88.68	89.58	81.13
Water	88.24	100	80.00	100
Overall accuracy	Overall accuracy = 85.95%, Kappa coefficient = 0.84		Overall accuracy = 84.32%, Kappa coefficient = 0.82

.

The error matrixes (Figure 16, Table 2) reveal that the two methods produce similar results, and only minor differences occur. The overall accuracy of our method with the ontology model is 85.95%, and the kappa coefficient is 0.84. The overall accuracy of the decision tree method without ontology model is 84.32%, and the kappa coefficient is 0.82 (see Table 2). Our method with ontology yields small improvements as it depends on the initial segmentation method. This again is based on the semantic rules used in the semantic classification process which validates the initial classification method furthermore, and some obvious classification errors may be corrected already within the following semantic classification step.

The producer’s accuracy of our method for all land-cover types except for Woodland and Grassland are higher than those based on the decision tree method without ontology, as shown in Table 1. The user’s accuracy of our method for Field, Grassland and Bare land are higher than the decision tree method as shown in Table 1. Given that the method employs semantic rules to restrict, it reduces misclassification to a certain extent. However, obvious misclassification instances between Building and Road exist because the two classes are spectrally too similar.

3.2. Discussion

The classification results reveal some small improvements of the accuracies when including ontology. However, the ontology model helps in understanding the complex structure of the overall GEOBIA framework, both for the human operators as well as for the software agents used. Even more importantly, the ontology enables the reuse of the general GEOBIA framework, makes GEOBIA assumptions explicit, and enables the operator to analyse the GEOBIA knowledge in great detail. The ontology model of image object features uses the GEOBIA structure as the upper level knowledge to be further extended. The land-cover ontology model is built based on the official Chinese Geographical Conditions Census Project and the upper level of LCCS. It only builds the decision tree ontology model and the semantic rule ontology model, respectively. Both models can be extended to realize the semantic understanding of various land cover categories. The process of image interpretation in the geographic domain—as opposed to, e.g., industrial imaging—is an expert process and many of the parameters need to be tuned depending on the problem domain [19]. We strongly believe that particularly the high degree of variance of natural phenomena in landscapes and potential regional idiosyncrasies can be managed well when formal ontologies serve as a central part the overall GEOBIA framework.

The ontology model for land-cover, image object features, and classifiers was formalised through the use of OWL and SWRL formal languages. In fact, the entire semantic network model was built in such a formalized way around the central element of the ontology model for object classification. The knowledge for building the model was acquired both from literature [24] as well as by using data mining techniques [48,49]. Although the experiments were so far carried out for one high resolution ZY-3 satellite image scene and can therefore not verify the transferability as such, the experiments clearly validated the feasibility of the methodology. The ontology part—as compared to the decision tree classification without using the ontology—yielded only minor statistical improvements in terms of accuracy for this particular image. Still, it validated the ontology to become the central part of a GEOBIA classification framework and a future standard methodology to handle knowledge for a certain domain or for several domains. The authors believe that this methodology can help to reduce the semantic gap which exists in performing image classification [15,19]. Image objects were classified by combining decision tree and semantic rules, which not only provide the classification result of the geographical objects, but also master the semantic information of the geographical entities, and realize the reuse of domain knowledge and the semantic network model. Our results are in line with Arvor et al. [19] who comprehensively portray GEOBIA ontologies for data discovery, automatic image interpretation, data interoperability, workflow management and data publication. In accordance with Blaschke et al. [15], we also want to emphasize the potential to bridge remote sensing and image processing methods with Geoinformatics/GIScience methods.

The authors should reemphasize that the methodology is knowledge-driven and shall be shared among experts so as to enhance and share. One obstacle here may be that the software framework used called FeatureStation is very powerful and successfully used in operational studies for the whole of China but it is currently only available in Chinese language and therefore not used internationally. The particular challenges of our method include the determination of an appropriate segmentation scale, the determination of the importance of the various features, the choice of the classifier, and the determination of parameters. Still, all four aspects mentioned are typical to any GEOBIA approach and the ontology provides a great stability for the determination of the parameters and for iterative testing. This study focuses on the implementation process of the method, and overall accuracy is used to evaluate the feasibility of the method. It should be pointed out that the determination of a study of systematic influences of the various elements of the GEOBIA framework on the influence of this method such as the segmentation method, the scale or the features used is simply described in this study, and for the problems and consequences of the different segmentation algorithms, the scale parameter and the feature selection methods we refer to the two recent publications of Ma et al. [50] and Li et al. [51].

4. Conclusions

The study has put forward an object-based semantic classification method for high resolution remote sensing imagery using ontology. It aimed to fully exploit the advantages of ontology to GEOBIA. A detailed workflow has been introduced that includes three major steps: ontology modelling, initial classification based on decision tree machine learning method, and semantic classification based on semantic rules. The structure of the GEOBIA framework was organized organically and expressed explicitly through the ontology. Operationally, the semantic relations were expressed in the OWL and SWRL formal languages with which the software can operate directly. Image objects were classified based on the ontology model by using a decision tree and semantic rules. It could be demonstrated that this ontology based GEOBIA framework can serve as an objective model and can enhance remote sensing image classification, particularly by enhancing the degree of automation and the reusability and transferability of the classification framework.

Nevertheless, building a comprehensive ontology is difficult and time-consuming, there is no single correct ontology for any domain, the potential applications of the ontology and the designer’s understanding and view of the domain will undoubtedly affect ontology design choices [33]. Domain experts should be involved in the construction of ontologies. We may also diagnose that, at least in the fields of remote sensing and image classification, ontologies are still rare and have not made their way into standard workflows. As discussed in the introduction section several existing studies have already demonstrated the potential of ontologies for particular domains or specific regional instances. Our study could clearly go one step further: we created a fully functional comprehensive reusable GEOBIA framework. Further in-depth studies may be required to: (a) improve and refine the GEOBIA ontology model; (b) build ontology models for new classifiers such as deep learning, random forests and random fern; and (c) investigate the automation and “geo-intelligence” potential [52] of the ontology-driven object-based semantic classification method.

Particularly through the highly successful GEOBIA conferences such as the 2016 conference organized in Enschede, the Netherlands, worldwide collaborations in ontology research in the field of remote sensing—reaching out to GIS and Geoinformatics—have begun. The authors promote to utilize existing ontologies and the Semantic Web as an extension of the World Wide Web that enables people to share content. It may serve as a web of data and has inspired and engaged scientists to create and share innovative semantic technologies and applications. Therefore, we encourage researchers and experts and to develop shared ontologies to allow for more domain specific ontologies, and to enhance the automation of GEOBIA.