Unsupervised Local Binary Pattern Histogram Selection Scores for Color Texture Classification

Abstract

These last few years, several supervised scores have been proposed in the literature to select histograms. Applied to color texture classification problems, these scores have improved the accuracy by selecting the most discriminant histograms among a set of available ones computed from a color image. In this paper, two new scores are proposed to select histograms: The adapted Variance score and the adapted Laplacian score. These new scores are computed without considering the class label of the images, contrary to what is done until now. Experiments, achieved on OuTex, USPTex, and BarkTex sets, show that these unsupervised scores give as good results as the supervised ones for LBP histogram selection.

1. Introduction

Texture classification is an active research topic in image processing and computer vision. It has received significant attention in many applications such as content based image retrieval, medical image analysis, face recognition, or biometrics. The texture classification approaches can typically be categorized into two subproblems [1,2]: The representation, which aims to characterize an image with a set of texture features, and the decision, which assigns this image to one of the available texture classes. This paper focuses on the first subproblem and particularly on feature space dimensionality reduction techniques. Many approaches perform a reduction of the feature space to transform high-dimensional data into a meaningful representation of reduced dimensionality [3,4,5]. By only retaining the most discriminant features, these approaches aim to improve the classification accuracy, while decreasing the processing time.

Dimensionality reduction techniques can be divided into two categories [6]. (1) Feature extraction builds a low dimensional subspace where the new features are usually combinations of the original features. The main drawback of this strategy is it requires to compute all candidate features during the classification stage to build the new feature space, which could be time-consuming; (2) Feature selection strategies select the most relevant original features. Hence, it just requires the computation of a reduced number of selected features during the classification stage. Among the feature selection techniques, we were particularly interested in those based on individual ranking. These algorithms rank the candidate features with respect to a score which measures their relevance. They are relatively inexpensive in computation time since no subspace procedure generation is used.

In the supervised context, the information about the class distribution is available. Supervised feature selection scores, such as the Fisher and the Supervised Laplacian scores, use the class labels to determine the relevance of each feature. However, it would be interesting to see if using a soft way to measure the similarity between images could be relevant. A soft value does not use any information about the class label of the images but measures the similarity in a subtle way, instead of being binary with just two values (same class or not). This may lead to powerful discriminating information since it should better reflect the geometric structure of the different classes. The Variance and the Laplacian scores measure the ability of a feature to keep the intrinsic data structure without considering any information about the class label of the images [7]. In accordance with this, they could be considered to be unsupervised. These two different scores, originally designed for the selection of features, have been successfully used in the context of image classification to select relevant features and improve the classification accuracy [8]. In this paper, we propose to see if the soft way to measure the similarity between images used in these two unsupervised scores is relevant for selecting histograms.

To describe the texture, the local patterns contained in an image are usually represented by histograms, like sum and difference histograms [9], histograms of equivalent patterns [10] or bag-of-words histograms [11]. A set of cross-channel histograms are then computed to represent a color texture. Local Binary Pattern (LBP) is a texture descriptor belonging to this scheme [12]. The LBP operator transforms an image by thresholding the levels of the P neighboring pixels around each pixel of the image, and coding the result as a binary number. Usually, the histogram of this LBP image is then used for texture analysis. Many authors have taken an interest in the reduction of this $(2^P=Q)$-dimensional LBP histogram in order to improve the texture classification performances [13]. Ojala et al. propose to consider the uniform LBP operator, where 59 discriminant pattern types (or bins) are a priori chosen among the $2^8=256$ available ones [14]. Mäenpää et al. consider a method based on beam search to select a reduced number of discriminant bins [15]. Boosting has become a very popular approach for feature selection and has been widely adopted for LBP feature selection in various tasks [13]. Liao et al. introduce the Dominant Local Binary Pattern (DLBP) that considers the most frequently occurred patterns to improve the recognition accuracy [16]. Because DLBP is only based on the pattern frequency, information about the type (label) of the selected patterns is lost. That is the reason why this texture descriptor has been later improved by labelling the most frequent patterns [17], like in the Labelled Dominant Local Binary Pattern (L-DLBP) [18], the Highest-Variance Dominant Local Binary Pattern (HV-DLBP) [19] or more recently in the Highest-Rank Dominant Local Binary Pattern (HR-DLBP) [20]. Guo et al. also propose a labelled model of the DLBP based on the Fisher separation criteria [21,22]. The most reliable and robust dominant bins are thus determined by considering intra-class similarity and inter-class dissimilarity.

Many other extensions or variants of the LBP operator have been proposed in recent decades for gray level images [12]. However, the extensions of this operator applied to color images remain relatively limited since 2002, wherein the Extended Opponent Color LBPs (EOCLBP) have been proposed by Pietikäinen et al. [13]. In EOCLBP, the LBP operator is applied on each color component of a given color space independently and also on pairs of color components according to a cross-channel strategy. This leads to extract nine different histograms, three within-component and six between-component LBP histograms, and it could be interesting to wonder whether all the information contained in these histograms is relevant to discriminate the textures. Paradoxically, reducing the dimensionality of LBP histograms is much less frequent in the framework of color texture analysis whereas the dimension of the feature space is higher. A first solution proposed by Chan et al. uses linear discriminant analysis to project high-dimensional color LBP bins into a discriminant space [23]. A second solution is proposed by Hussain et al., who exploit the complementarity of Histograms of Oriented Gradients [24], Local Binary Patterns, and Local Ternary Patterns [25] and apply partial least squares to resolve their visual object detection problem [26]. More recently, Porebski et al. propose a different approach which selects, out of the nine LBP histograms extracted from a color texture, those which are the most discriminant [27]. This strategy, which selects histograms in their entirety, fundamentally differs from all the previous LBP selection approaches which select the bins of the LBP histograms or project them into a discriminant space. To evaluate the relevance of the LBP histograms, Porebski et al. propose a supervised approach where an Intra-Class Similarity score ($ICS$-score) is computed for each histogram. This score is based on a measure of the histogram ability to characterize the similarity of the textures within each different class. Inspired by this approach, Kalakech et al. propose another score (the $ASL$-score) based on the supervised Laplacian score designed for feature ranking and selection [28]. In [29], histogram selection and bin selection schemes have been extended to the multi-color space domain and compared each other in the framework of color texture classification. It has been shown that the classification accuracy reached thanks to histogram selection is slightly higher than the accuracy provided by a bin selection, with a similar classification computation time. The encouraging results obtained with the two supervised $ICS$ and $ASL$-score lead us to propose in this paper two new histogram selection scores: The adapted Variance ($AV$-score) and the adapted Laplacian ($AL$-score) scores. As the names suggest, these scores are respectively adapted from the unsupervised Variance and Laplacian scores which have been originally designed for the selection of features and which use a soft way to measure the similarity between images. In this paper, we propose to extend these scores in order to rank and select LBP histograms extracted from a color image.

First, the traditional unsupervised feature selection scores are presented in Section 2. The corresponding adapted histogram selection scores are then detailed in Section 3 and the LBP histogram selection approach is described in Section 4. In order to compare these two new scores each other and with the results of the state of the art, experiments are performed on benchmark and widely used databases in Section 5.

2. Feature Selection Scores

In the feature selection context, we dispose a dataset of N color texture images represented in a D-dimensional feature space. We denote $\mathbf{X}$, the associated data matrix where $x_i^r$ is the rth feature value ($r=1,\dots ,D$) of the ith color image $I_i$ ($i=1,\dots ,N$).

$$\mathbf{X}=\left[\begin{array}{ccccc}{x}_1^1& \dots & x_1^r& \dots & x_1^D\\ \dots & \dots & \dots & \dots & \dots \\ x_i^1& \dots & x_i^r& \dots & x_i^D\\ \dots & \dots & \dots & \dots & \dots \\ x_N^1& \dots & x_N^r& \dots & x_N^D\end{array}\right]=\left[\begin{array}{c}{\mathbf{x}}_1\\ \dots \\ {\mathbf{x}}_i\\ \dots \\ {\mathbf{x}}_N\end{array}\right]=\left[\begin{array}{c}{\mathbf{f}}^1\dots {\mathbf{f}}^r\dots {\mathbf{f}}^D\end{array}\right].$$

Each of the N rows of the matrix $\mathbf{X}$ represent a color texture ${\mathbf{x}}_i=\left(x_i^1,\dots ,x_i^D\right)$ ∈ ${\mathbb{R}}^D$, while each of the D columns of $\mathbf{X}$ define the feature ${\mathbf{f}}^r={\left(x_1^r,\dots ,x_N^r\right)}^T$ ∈ ${\mathbb{R}}^N$.

2.1. Unsupervised Feature Selection Scores

In the unsupervised context, the Variance and the Laplacian scores are usually used to select features [7].

2.1.1. Variance Score

The variance score $V^r$ used to evaluate the relevance of each feature ${\mathbf{f}}^r$ is defined by:

$$V^r=\frac{1}{N}\sum _{i=1}^N{\left(x_i^r-{\mu }^r\right)}^2,$$

(1)

where ${\mu }^r$ is the mean for all images of the feature ${\mathbf{f}}^r$: ${\mu }^r=\frac{1}{N}{\sum }_{i=1}^N{x}_i^r$.

The features are sorted according to the decreasing order of $V^r$ in order to select the most relevant ones, assuming that the feature with the highest variance is the most discriminant one.

2.1.2. Laplacian Score

Rather than measuring the data dispersion along a feature axis, the Laplacian score examines the local properties of the data. He et al. propose to compute the Laplacian score $L^r$ of a feature ${\mathbf{f}}^r$ as [7]:

$$L^r=\frac{{\sum }_{i=1}^N{\sum }_{j=1}^N{\left(x_i^r-x_j^r\right)}^2{s}_{ij}}{{\sum }_{i=1}^N{\left(x_i^r-{\overline{f}}^r\right)}^2{d}_i},$$

(2)

where:

• ${\left(x_i^r-x_j^r\right)}^2$ is the squared Euclidean distance between the rth feature of two images $I_i$ and $I_j$,
• $s_{ij}$ is the similarity measure between $I_i$ and $I_j$ using all the input feature space composed by the D features. It is defined by: $s_{ij}=exp\left(-\frac{{∥{\mathbf{x}}_i-{\mathbf{x}}_j∥}^2}{2t^2}\right)$, where ${∥{\mathbf{x}}_i-{\mathbf{x}}_j∥}^2$ represents the squared Euclidean distance between ${\mathbf{x}}_i$ and ${\mathbf{x}}_j$ in the D-dimensional initial feature space [30,31]. The parameter t has to be tuned in order to represent the local dispersion of the data [32],
• $d_i$ represents a local density measure defined by: $d_i={\sum }_{j=1}^N{s}_{ij}$,
• and ${\overline{f}}^r$ is the weighted feature average: ${\overline{f}}^r=\frac{{\sum }_{i=1}^N{x}_i^r{d}_i}{{\sum }_{i=1}^N{d}_i}$.

The different features are sorted according to the ascending order of $L^r$ in order to select the most relevant ones: The features which respect the pre-defined graph structure minimize the numerator of $L^r$ and those with a large variance maximize the denominator.

These unsupervised feature selection scores have originally been designed for feature selection. However, in the framework of color texture characterization, histograms are widely used as texture descriptors, like the LBP histograms [13]. Because the number of those histograms can be high and problematic for classification purpose, it would be interesting to select the most discriminant ones to improve the classification performances. For this purpose, we propose to adapt the traditional feature selection scores used for feature ranking and selection, in order to rank and select histograms.

3. Histogram Selection Scores

In the histogram selection context, we dispose a dataset of N color textures images. Each image $I_i$ ($i=1,\dots ,N$) is characterized by D histograms. The whole data is summarized by the matrix $\mathbf{H}$ as:

$$\mathbf{H}=\left[\begin{array}{ccccc}{\mathbf{h}}_1^1& \dots & {\mathbf{h}}_1^r& \dots & {\mathbf{h}}_1^D\\ \dots & \dots & \dots & \dots & \dots \\ {\mathbf{h}}_i^1& \dots & {\mathbf{h}}_i^r& \dots & {\mathbf{h}}_i^D\\ \dots & \dots & \dots & \dots & \dots \\ {\mathbf{h}}_N^1& \dots & {\mathbf{h}}_N^r& \dots & {\mathbf{h}}_N^D\end{array}\right]=\left[\begin{array}{c}{\mathbf{h}}_1\\ \dots \\ {\mathbf{h}}_i\\ \dots \\ {\mathbf{h}}_N\end{array}\right]=\left[\begin{array}{c}{\mathbf{h}}^1\dots {\mathbf{h}}^r\dots {\mathbf{h}}^D\end{array}\right]$$

where ${\mathbf{h}}_i^r$ is the rth histogram computed from the ith color texture image $I_i$. It is defined by ${\mathbf{h}}_i^r=\left(h_i^r(1),\dots ,h_i^r(k),\dots ,h_i^r(Q)\right)$ where Q is the histogram bin number.

The ith row of $\mathbf{H}$ represents a set of D histograms ${\mathbf{h}}_i$ corresponding to the image $I_i$ and whose dimension is $(D\times Q)$. For each column, ${\mathbf{h}}^r={\left[{\mathbf{h}}_1^r\dots {\mathbf{h}}_i^r\dots {\mathbf{h}}_N^r\right]}^T$ regroups the values of the rth histogram across the N images.

The histogram selection scheme evaluates each histogram ${\mathbf{h}}^r$ in order to select the most discriminant one among the D candidate histograms. For this purpose, we propose to adapt the feature selection scores, presented in Section 2, in order to define histogram selection scores. Distance and similarity measures are two critical terms used for feature selection. Distance measures are low when the images are close to each other, contrary to similarity measures whose highest value indicates that the considered images are similar. To adapt the traditional feature selection scores to rank and select histograms, it is necessary to consider either a distance measure between histograms or a similarity measure between histograms depending on whether the term to adapt has to be maximized or minimized.

Several measures of similarity and distance between histograms have been used in computer vision and pattern recognition [33]. Since the objective of this paper is to show the interest of the proposed scores, we retain two simple measures, the histogram intersection as similarity measure and the Jeffrey distance as distance measure: the histogram intersection is considered to adapt the similarity term $s_{ij}$ which has to be maximized (the kernel is maximized when the images are similar) and the Jeffrey distance is used to extend the Euclidean distance which has to be minimized for similar images.

The intersection between the histograms extracted from two images $I_i$ and $I_j$ is defined as follows:

$$S({\mathbf{h}}_i,{\mathbf{h}}_j)=\sum _{k=1}^{Q\times D}min(h_i(k),h_j(k)).$$

(3)

The result of the intersection is the number of pixels of the first image that have a corresponding pixel in the second image which has the same characteristic (the same specific pattern in the case of LBP histograms). So the more the considered images are similar, the higher the histogram intersection is. The histograms being normalized by the number of pixels in the image, the value of this measure varies between 0 and 1.

The Jeffrey distance between the histograms of two images $I_i$ and $I_j$ is defined as follows:

$$J({\mathbf{h}}_i,{\mathbf{h}}_j)=\sum _{k=1}^{Q\times D}{h}_i(k)log\left(\frac{h_i(k)}{\frac{h_i(k)+h_j(k)}{2}}\right)+h_j(k)log\left(\frac{h_j(k)}{\frac{h_i(k)+h_j(k)}{2}}\right).$$

(4)

As all distance measures, the value of the Jeffrey distance is low when the images are close to each other in the histogram space.

In order to clarify the adaptation of the different scores to histogram selection, we summarize the terms and the scores used in this section in

Table 1. Summary of the terms and the scores used in feature selection and their corresponding histogram selection adaptation.

	Feature Selection	Histogram Selection
Dataset	Dataset of N color texture images defined in a D-dimensional feature space	Dataset of N color texture images defined in a $(Q\times D)$-dimensional histogram space
Data matrix	$\mathbf{X}=\left(x_i^r\right)$; $i=1,\dots ,N$; $r=1,\dots ,D$ $x_i^r$ is the rth feature value of the ith image $I_i$	$\mathbf{H}=\left({\mathbf{h}}_i^r\right)$; $i=1,\dots ,N$; $r=1,\dots ,D$ ${\mathbf{h}}_i^r$ is the rth histogram extracted from the ith image $I_i$
Row	${\mathbf{x}}_i=\left(x_i^1,\dots ,x_i^D\right)$	${\mathbf{h}}_i=\left[{\mathbf{h}}_i^1\dots {\mathbf{h}}_i^r\dots {\mathbf{h}}_i^D\right]$ with ${\mathbf{h}}_i^r=\left(h_i^r(1),\dots ,h_i^r(k),\dots ,h_i^r(Q)\right)$
Column	${\mathbf{f}}^r={\left(x_1^r,\dots ,x_N^r\right)}^T$	${\mathbf{h}}^r={\left[{\mathbf{h}}_1^r\dots {\mathbf{h}}_i^r\dots {\mathbf{h}}_N^r\right]}^T$
Selection	The most discriminant features ${\mathbf{f}}^r$ among the D available ones	The most discriminant histograms ${\mathbf{h}}^r$ among the D available ones
Distance	${\left(x_i^r-x_j^r\right)}^2$ is the squared Euclidean distance between the two images $I_i$ and $I_j$ using the considered feature ${\mathbf{f}}_r$	$J^2({\mathbf{h}}_i^r,{\mathbf{h}}_j^r)$ is the squared Jeffrey distance between the two images $I_i$ and $I_j$ using the considered histogram ${\mathbf{h}}^r$ $J({\mathbf{h}}_i^r,{\mathbf{h}}_j^r)={\sum }_{k=1}^Q{h}_i^r(k)log\left(\frac{h_i^r(k)}{\frac{h_i^r(k)+h_j^r(k)}{2}}\right)+h_j^r(k)log\left(\frac{h_j^r(k)}{\frac{h_i^r(k)+h_j^r(k)}{2}}\right)$
Similarity	$s_{ij}$ evaluates the similarity between the images $I_i$ and $I_j$ in the D-dimensional input space $s_{ij}=exp\left(-\frac{{∥{\mathbf{x}}_i-{\mathbf{x}}_j∥}^2}{2t^2}\right)$	$S({\mathbf{h}}_i,{\mathbf{h}}_j)$ evaluates the similarity between the images $I_i$ and $I_j$ in the $(Q\times D)$-dimensional input space using the histogram intersection $S({\mathbf{h}}_i,{\mathbf{h}}_j)={\sum }_{k=1}^{Q\times D}min\left(h_i(k),h_j(k)\right)$
Mean	${\mu }^r=\frac{{\sum }_{i=1}^N{x}_i^r}{N}$	${\overline{\mathbf{h}}}^r=\left({\overline{h}}^r(1),\dots ,{\overline{h}}^r(k),\dots ,{\overline{h}}^r(Q)\right)$ with ${\overline{h}}^r(k)=\frac{1}{N}{\sum }_{i=1}^N{h}_i^r(k)$
Variance Score	$V^r=\frac{1}{N}{\sum }_{i=1}^N{\left(x_i^r-{\mu }^r\right)}^2$	$A{V}^r=\frac{1}{N}{\sum }_{i=1}^N{J}^2\left({\mathbf{h}}_i^r,{\overline{\mathbf{h}}}^r\right)$
Degree	$d_i={\sum }_{j=1}^N{s}_{ij}$	$D_i={\sum }_{j=1}^{N}S({\mathbf{h}}_i,{\mathbf{h}}_j)$
Weighted average	${\overline{f}}^r=\frac{{\sum }_{i=1}^N{x}_i^r{d}_i}{{\sum }_{i=1}^N{d}_i}$	${\overline{\mathbf{a}}}^r=\left({\overline{a}}^r(1),\dots ,{\overline{a}}^r(k),\dots ,{\overline{a}}^r(Q)\right)$ with ${\overline{a}}^r(k)=\frac{{\sum }_{i=1}^N{h}_i^r(k)D_i}{{\sum }_{i=1}^N{D}_i}$
Laplacian Score	$L^r=\frac{{\sum }_{i=1}^N{\sum }_{j=1}^N{\left(x_i^r-x_j^r\right)}^2{s}_{ij}}{{\sum }_{i=1}^N{\left(x_i^r-{\overline{f}}^r\right)}^2{d}_i}$	$A{L}^r=\frac{{\sum }_{i=1}^N{\sum }_{j=1}^N{J}^2({\mathbf{h}}_i^r,{\mathbf{h}}_j^r)S({\mathbf{h}}_i,{\mathbf{h}}_j)}{{\sum }_{i=1}^N{J}^2({\mathbf{h}}_i^r,{\overline{\mathbf{a}}}^r)D_i}$

where formulas are applied to evaluate the score of the rth histogram. The left column groups feature selection terms while the right one summarizes the corresponding histogram selection adaptation. Readers can refer to this table while reading the next section.

3.1. Adapted Variance Score

Using the Jeffrey distance defined in Equation (4), we extend the Variance score of Equation (1) in order to select histograms rather than features. The Adapted Variance score $A{V}^r$ of the histogram ${\mathbf{h}}^{\mathbf{r}}$ is defined as follows:

$$A{V}^r=\frac{1}{N}\sum _{i=1}^N{J}^2\left({\mathbf{h}}_i^r,{\overline{\mathbf{h}}}^r\right),$$

(5)

where ${\overline{\mathbf{h}}}^r$ is the mean histogram that is evaluated by averaging all the bins of the histogram ${\mathbf{h}}^r$ across the N images: ${\overline{\mathbf{h}}}^r=\left({\overline{h}}^r(1),\dots ,{\overline{h}}^r(k),\dots ,{\overline{h}}^r(Q)\right)$, with ${\overline{h}}^r(k)=\frac{1}{N}{\sum }_{i=1}^N{h}_i^r(k)$.

The histograms are sorted according to the decreasing order of $A{V}^r$ in order to select the most relevant ones.

3.2. Adapted Laplacian Score

Using the Jeffrey distance and the intersection similarity measure defined in Equations (3) and (4), we extend the Laplacian score of Equation (2) in order to select the most discriminant histograms. The Adapted Laplacian score $A{L}^r$ of the histogram ${\mathbf{h}}^{\mathbf{r}}$ is defined as follows:

$$A{L}^r=\frac{{\sum }_{i=1}^N{\sum }_{j=1}^N{J}^2({\mathbf{h}}_i^r,{\mathbf{h}}_j^r)S({\mathbf{h}}_i,{\mathbf{h}}_j)}{{\sum }_{i=1}^N{J}^2({\mathbf{h}}_i^r,{\overline{\mathbf{a}}}^r)D_i}.$$

(6)

The degree $D_i$ of the image $I_i$ is defined by: $D_i={\sum }_{j=1}^{N}S({\mathbf{h}}_i,{\mathbf{h}}_j)$ and ${\overline{\mathbf{a}}}^r$ is the weighted histogram average: ${\overline{\mathbf{a}}}^r=\left({\overline{a}}^r(1),\dots ,{\overline{a}}^r(k),\dots ,{\overline{a}}^r(Q)\right)$, with ${\overline{a}}^r(k)=\frac{{\sum }_{i=1}^N{h}_i^r(k)D_i}{{\sum }_{i=1}^N{D}_i}$.

As for feature selection using the Laplacian score, the histograms are sorted according to the ascending order of $A{L}^r$ in order to select the most relevant ones.

4. LBP Histogram Selection for Color Texture Classification

The adapted scores previously presented are used in a LBP histogram selection approach described in this section (see Section 4.2). The candidate color LBP histograms are first presented.

4.1. Candidate Color Texture Descriptors

The LBP operator is one of the most successful descriptor used to characterize texture images due to its ease of implementation, its invariance to monotonic illumination changes, and its low computational complexity. Many variants of the original LBP operator have been proposed in the literature since Ojala’s original definition [12]. The goal of this paper being to reveal the relevance of the proposed histogram selection scores, no further sophisticated texture descriptors are needed. That is the reason why the color textures are here characterized thanks to the EOCLBP histograms, which are a simple extension to color of the original LBP operator. Obviously, the classification results are expected to be improved using more elaborated descriptors, such as the Improved Opponent Color LBP [34] or the Median Robust Extended LBP for example [35], which is a gray level descriptor that has obtained the best overall performance on thirteen texture image sets and which could be extended to color.

To compute the EOCLBP histograms, each image is first coded in a 3-dimensional color space, denoted here $C_1{C}_2{C}_3$. The $D=9$ LBP histograms are then computed from the so-coded images: Three within-component LBP histograms ($(C_1,C_1)$, $(C_2,C_2)$, and $(C_3,C_3)$) and six between-component LBP histograms ($(C_1,C_2)$, $(C_2,C_1)$, $(C_1,C_3)$, $(C_3,C_1)$, $(C_2,C_3)$, and $(C_3,C_2)$) are extracted from each image. As do Ojala et al. when they introduce the original LBP operator, the $3\times 3$ pixel neighborhood ($P=8$ neighbors) is here considered. A color texture is thus represented by a ($9\times 256$)-dimensional feature space.

It is well-known that the performance of a classifier is generally dependent on the dimension of the feature subspace due to the curse of dimensionality [36]. To reach a satisfying classification accuracy while decreasing the computation time, we propose to reduce the number of candidate LBP histograms by selecting the most discriminating ones thanks to the histogram selection scores previously presented.

4.2. Histogram Selection

To evaluate a supervised color texture classification scheme, it is usual to divide the considered database into a learning and a testing image subset. The learning subset is used to train the classifier during the learning stage, whereas the testing subset is used during the classification stage to evaluate the performances of the proposed method. In the histogram selection framework, the learning stage aims to build a low dimensional discriminating subspace thanks to labelled or unlabelled training data.

Different models are proposed in order to evaluate the relevance of the candidate subspaces [37]. The wrapper model uses the classification accuracy as discriminating power of the candidate subspaces. When a classifier such as the nearest neighbor is considered, it involves to decompose the learning subset into a training and a validation subsets. Although this model is time consuming and classifier-dependent, it gives good results and determines easily the dimension of the selected subspace by searching the best classification accuracy. On the contrary, filter models evaluate the relevance of the candidate subspaces without classifying the images. They are less time consuming but the determination of the dimension of the subspace to be selected is not so easy. To obtain a good compromise between dimension selection, computation time and classification result, embedded models are preferred [38]. These approaches combine a filter model to determine the most discriminating subspaces at different dimensions and a wrapper model to determine the dimension of the selected subspace [6].

The approach used in this paper is an embedded histogram selection scheme which requires to split up the initial image database in a training, a validation and a testing image subset, according to a holdout decomposition. During the learning stage, candidate histograms are generated from training images and ranked with respect to a score which measures the efficiency of each candidate histogram. This score can be computed without considering the class label of the images like the unsupervised selection $AV$-score and $AL$-score or by taking the information about the class distribution into account, like the $ASL$-score and the $ICS$-score do.

Once the score has been computed for each of the D candidate histograms, a ranking is performed. The candidate subspaces—composed, at the first step, of the histogram with the best score, at the second step, of the two first ranked histograms and so on—are then evaluated to determine the relevant histogram subspace. For this purpose, a classifier operates in each candidate subspace in order to classify the validation images. For each subspace dimension d, the classification accuracy is estimated as the percentage of the validation images that have been correctly classified. This rate of well-classified validation images is denoted $R_d$.

The dimension $\widehat{d}$ of the selected subspace is the one for which the value of $R_d$ is the highest:

$$\widehat{d}=Q\times \underset{1\le d\le D}{argmax}{R}_d.$$

(7)

During the classification stage, the relevant histograms previously selected are computed for each testing image and compared to the training images in the selected histogram subspace to determine the testing image label. The purpose of this paper being to show the contribution of the two new histogram selection scores, independently of the considered classifier, its parameters and its metric, the nearest neighbor classifier associated with the histogram intersection as a similarity measure is here considered.

5. Experiments

In this section, the proposed histogram selection scores are compared thanks to three benchmark color texture image sets: Outex-TC-00013, USPTex, and NewBarkTex.

Outex-TC-00013 is composed of 68 color texture images acquired under controlled conditions by a 3-CCD digital color camera and the size of which is $746\times 538$ pixels [39]. Each of these 68 textures is split up into 20 $128\times 128$ disjoint sub-images. Among these 1360 sub-images, 680 are used for the training subset and the remaining 680 are considered as testing images. The Outex-TC-00013 image test suite can be downloaded at http://www.outex.oulu.fi/index.php?page=classification.

USPTex set is a more recent database [40]. It contains 191 natural color textures acquired under an unknown but fixed light source. As for Outex-TC-00013, these images are split up into $128\times 128$ disjoint sub-images. Since the original image size is here $512\times 384$ pixels, this makes a total of 12 sub-images by a texture. For our experiments, this initial dataset of 2292 sub-images is split up in order to build a training and a testing image subset: 6 images are considered for the training and the 6 others are used as testing images. This decomposition is available at https://www-lisic.univ-littoral.fr/~porebski/USPtex.zip.

The Barktex database includes six tree bark classes, with 68 images per class [41]. Even if the number of classes of this database is limited to six, the textures of these different classes are close to each other and their discrimination is not easy. To build the NewBarkTex set, a region of interest, centered on the bark and whose size is $128\times 128$ pixels, is first defined. Then, four sub-images whose size is $64\times 64$ pixels are extracted from each region. We thus obtain a set of $68\times 4=272$ sub-images per class. To ensure that color texture images used for the training and the testing images are less correlated as possible, the four sub-images extracted from a same original image all belong either to the training subset or to the testing one [42]: 816 images are thus used as training images and the remaining 816 as testing images. The NewBarkTex image test suite can be downloaded at https://www-lisic.univ-littoral.fr/~porebski/NewBarkTex.zip.

These sets do not require to consider specific illuminant or rotation invariant texture descriptors since the goal of this paper is to reveal the contribution of the proposed histogram selection scores independently of the texture descriptor invariance to the observation conditions.

Let us note that the considered texture benchmark databases are composed of only two image subsets according to a holdout evaluation method, whereas the considered histogram selection scheme needs three subsets as explained in Section 4.2. We thus propose to use one subset as the training subset and the second both as the validation and testing subset to evaluate the performances of the proposed scores. Therefore, the dimensionality of the selected feature space will be ideally determined, and the classification results can be interpreted as optimistic. This solution was nevertheless chosen in order to achieve the comparison with other works using the same split into training and testing subsets.

Moreover, in order to evaluate the impact of the used color space, four color spaces are considered for experiments: $RGB$, $YUV$, $I_1{I}_2{I}_3$, and $HSV$. These color spaces are respectively representative of the four color space families (the primary spaces, the luminance-chrominance spaces, the independent color component spaces, and the perceptual spaces) and do not require to know illumination conditions like the $Lab$ color space for example [4].

Section 5.1 presents a comparison of the performances achieved by the proposed histogram selection scores. An analysis of the histogram rank is then done (cf. Section 5.2). Finally, in Section 5.3, the classification results obtained by the proposed approach are compared with the state of the art.

5.1. Comparison of the Histogram Selection Scores

In this section, four histogram selection scores are compared on Outex-TC-00013, USPTex, and NewBarkTex sets:

• the unsupervised Adapted Variance score ($AV$-score),
• the unsupervised Adapted Laplacian score ($AL$-score),
• the Adapted Supervised Laplacian score ($ASL$-score) proposed by Kalakech [28],
• and the supervised Intra-Class Similarity score ($ICS$-score) proposed by Porebski [27].

Figure 1, Figure 2 and Figure 3 show the rate $R_d$ of well-classified validation images according to the number d of ranked histograms on Outex-TC-00013, USPTex, and NewBarkTex sets, respectively, and for each considered color space.

These figures show that the accuracy obtained thanks to the unsupervised $AL$-score globally outperforms the results obtained by the $AV$-score, for the three databases and whatever the considered color space. In the same way, the $ASL$-score outperforms the $ICS$-score in the supervised context. These results confirm the high performances obtained thanks to the Laplacian scores in the context of feature selection [8]. For histogram selection, the interest of the similarity term to capture the intrinsic properties of the data is also demonstrated.

These figures also show that the $ASL$-score globally gives the highest accuracy, followed very closely by the unsupervised $AL$-score. These scores reach a high accuracy with a lower dimensional histogram subspace. The unsupervised $AL$-score, which is computed without considering the class label of the images, globally outperforms the supervised $ICS$-score, which takes the information about the class distribution into account. This confirms again the relevance of the similarity matrix used in the Laplacian scores to perform the selection.

Table 2. Accuracy $R_{\widehat{d}}$ (%) reached with the $\widehat{d}$-dimensional selected local binary pattern (LBP) histogram subspace, according to the different supervised and unsupervised scores on the Outex-TC-00013 set (the dimension of the histogram space is $D\times Q=9\times 256$ without selection).

	$\mathit{AV}$		$\mathit{AL}$		$\mathit{ASL}$		$\mathit{ICS}$		Without
	Score		Score		Score		Score		Selection
	${\mathit{R}}_{\widehat{\mathit{d}}}$	$\widehat{\mathit{d}}$	${\mathit{R}}_{\widehat{\mathit{d}}}$	$\widehat{\mathit{d}}$	${\mathit{R}}_{\widehat{\mathit{d}}}$	$\widehat{\mathit{d}}$	${\mathit{R}}_{\widehat{\mathit{d}}}$	$\widehat{\mathit{d}}$	$\mathit{R}$
$RGB$	93.25%	8	$\boxed{\mathbf{93.38}\mathbf{\%}}$	8	$\boxed{\mathbf{93.38}\mathbf{\%}}$	8	92.94%	9	92.94%
$YUV$	89.56%	9	91.03%	7	91.03%	7	89.56%	9	89.56%
$I_1{I}_2{I}_3$	88.67%	8	88.82%	8	88.97%	6	88.97%	8	88.68%
$HSV$	90.44%	9	91.91%	5	91.91%	5	91.03%	8	90.44%

,

Table 3. Accuracy $R_{\widehat{d}}$ (%) reached with the $\widehat{d}$-dimensional selected LBP histogram subspace, according to the different supervised and unsupervised scores on the USPTex set (the dimension of the histogram space is $D\times Q=9\times 256$ without selection).

	$\mathit{AV}$		$\mathit{AL}$		$\mathit{ASL}$		$\mathit{ICS}$		Without
	Score		Score		Score		Score		Selection
	${\mathit{R}}_{\widehat{\mathit{d}}}$	$\widehat{\mathit{d}}$	${\mathit{R}}_{\widehat{\mathit{d}}}$	$\widehat{\mathit{d}}$	${\mathit{R}}_{\widehat{\mathit{d}}}$	$\widehat{\mathit{d}}$	${\mathit{R}}_{\widehat{\mathit{d}}}$	$\widehat{\mathit{d}}$	$\mathit{R}$
$RGB$	89.53%	9	90.92%	5	91.27%	4	90.58%	7	89.53%
$YUV$	76.79%	9	$\boxed{\mathbf{93.19}\mathbf{\%}}$	3	$\boxed{\mathbf{93.19}\mathbf{\%}}$	3	$\boxed{\mathbf{93.19}\mathbf{\%}}$	3	76.79%
$I_1{I}_2{I}_3$	75.31%	9	92.06%	3	92.06%	3	92.06%	3	75.31%
$HSV$	83.25%	9	90.40%	3	90.40%	3	88.92%	5	83.35%

and

Table 4. Accuracy $R_{\widehat{d}}$ (%) reached with the $\widehat{d}$-dimensional selected LBP histogram subspace, according to the different supervised and unsupervised scores on the NewBarkTex set (the dimension of the histogram space is $D\times Q=9\times 256$ without selection).

	$\mathit{AV}$		$\mathit{AL}$		$\mathit{ASL}$		$\mathit{ICS}$		Without
	Score		Score		Score		Score		Selection
	${\mathit{R}}_{\widehat{\mathit{d}}}$	$\widehat{\mathit{d}}$	${\mathit{R}}_{\widehat{\mathit{d}}}$	$\widehat{\mathit{d}}$	${\mathit{R}}_{\widehat{\mathit{d}}}$	$\widehat{\mathit{d}}$	${\mathit{R}}_{\widehat{\mathit{d}}}$	$\widehat{\mathit{d}}$	$\mathit{R}$
$RGB$	73.16%	9	$\boxed{\mathbf{81.37}\mathbf{\%}}$	4	$\boxed{\mathbf{81.37}\mathbf{\%}}$	4	$\boxed{\mathbf{81.37}\mathbf{\%}}$	4	73.16%
$YUV$	71.81%	9	79.17%	7	79.17%	7	79.17%	7	71.81%
$I_1{I}_2{I}_3$	71.68%	9	79.41%	7	79.41%	7	79.41%	7	71.69%
$HSV$	70.59%	9	81%	3	81%	3	81%	3	70.59%

show the accuracies $R_{\widehat{d}}$ obtained with the $\widehat{d}$-dimensional selected LBP histogram subspaces, by using the different supervised and unsupervised scores on Outex-TC-00013, USPTex, and NewBarkTex sets, respectively. The accuracy reached without performing any color LBP histogram selection is also presented. The bold values represent the best rates obtained with each color space and the boxed values indicate the best rate obtained for each color texture set.

These tables confirm the interest of selecting LBP histograms: The selection improves the classification accuracy by on average 0.52% on OuTex, 7.70% on USPTex and 6.32% on BarkTex, while reducing the number of considered histograms. We can also see that the performances reached thanks to the different scores are very close to each other, especially for the OuTex and USPTex databases. For the color space that gives the best rates ($RGB$ for Outex-TC-00013 and NewBarkTex and $YUV$ for USPTex), several scores give the higher performances and the $ASL$ and the $AL$ scores always appear among the best scores.

For NewBarkTex which is a more challenging set, the $AL$, $ASL$, and $ICS$ scores give exactly the same best accuracy with the same optimal dimension. The difference between these three scores appears more for subspaces with a little dimension: From Figure 3, we can notice that the $ASL$ and the $AL$ scores seek faster the better histograms specially for the $YUV$ and $I_1{I}_2{I}_3$ color spaces.

It is also interesting to notice that the unsupervised $AL$-score appears among the best scores 10 times out of 12. It outperforms the other unsupervised $AV$-score and even the supervised $ICS$ score. Its performances are remarkable since they are similar or very close to those reached by $ASL$ even though it does not consider the class label of the images.

5.2. Comparison of the Histogram Ranks

In this section, an analysis of the histogram ranking is done.

Table 5. Histogram ranks using the proposed scores with the different color spaces and for the three databases.

		OuTex	USPTex	BarkTex
	$AV$-score	2 4 3 6 8 7 1 5 9	5 4 6 8 7 9 2 3 1	3 7 6 8 4 2 5 1 9
	$AL$-score	9 1 5 8 7 6 3 4 2	1 3 2 9 7 8 6 4 5	9 1 5 2 4 8 6 7 3
$RGB$	$ASL$-score	9 1 5 8 7 6 4 3 2	1 2 3 7 4 9 6 8 5	9 5 1 2 4 8 6 7 3
	$ICS$-score	8 7 1 9 5 3 4 2 6	3 1 2 8 7 9 4 5 6	9 1 5 2 8 4 6 7 3
	$AV$-score	8 4 6 2 7 3 1 9 5	8 7 9 4 5 6 1 3 2	8 6 4 7 2 3 9 5 1
	$AL$-score	5 9 1 3 7 6 2 4 8	3 2 1 4 5 6 7 9 8	3 1 2 5 9 7 4 6 8
$YUV$	$ASL$-score	1 9 5 6 8 3 7 2 4	3 2 1 4 5 6 9 7 8	3 2 7 4 1 5 9 6 8
	$ICS$-score	3 6 7 8 2 1 4 9 5	3 2 1 5 4 6 9 7 8	3 2 7 4 1 5 9 6 8
	$AV$-score	8 6 7 4 3 2 1 5 9	8 7 9 5 6 4 1 2 3	8 6 4 7 2 5 3 9 1
	$AL$-score	9 5 1 2 4 3 7 6 8	3 1 2 5 4 6 9 7 8	3 2 1 5 9 7 4 6 8
$I_1{I}_2{I}_3$	$ASL$-score	1 9 5 6 8 2 3 4 7	2 3 1 6 4 5 9 7 8	1 3 2 5 9 7 4 6 8
	$ICS$-score	2 4 3 6 7 8 1 9 5	3 2 1 5 4 6 9 8 7	3 2 7 4 1 5 9 6 8
	$AV$-score	3 2 6 8 7 4 1 5 9	6 4 7 9 5 8 1 2 3	8 7 2 4 6 3 1 9 5
	$AL$-score	9 5 1 7 4 3 2 8 6	3 2 1 8 7 4 5 9 6	5 9 1 4 2 3 6 7 8
$HSV$	$ASL$-score	1 5 9 8 6 7 4 3 2	2 3 1 7 4 9 8 6 5	5 1 9 4 2 3 6 7 8
	$ICS$-score	7 8 6 1 3 4 9 5 2	3 2 7 4 1 8 5 9 6	5 1 9 6 2 4 3 8 7

shows the histogram ranking obtained thanks the considered scores on Outex-TC-00013, USPTex and NewBarkTex sets. The numbers 1, 2 and 3 represent the three within-component LBP histograms ($(C_1,C_1)$, $(C_2,C_2)$, and $(C_3,C_3)$), and the six between-component LBP histograms ($(C_1,C_2)$, $(C_2,C_1)$, $(C_1,C_3)$, $(C_3,C_1)$, $(C_2,C_3)$, and $(C_3,C_2)$) are respectively numbered 4, 5, 6, 7, 8, and 9.

Each row of this table shows the histogram ranking in the considered color space using the specified histogram selection score, for the three image sets. For example the first row shows that, in the RGB color space, using the $AV$-score and the OuTex database, the first selected histogram is the number 2 ($(C_2,C_2)$), followed by the histogram 4 ($(C_1,C_2)$), ... and finally the histogram 9 ($(C_3,C_2)$) is the last selected. The bold values correspond to the selected histogram subspace for which the best accuracy is achieved for each of the three color texture sets.

This table shows that the histogram ranking is very variable according to the considered color space or score. This clearly shows the interest of performing a histogram selection, since we can not a priori judge the most relevant histogram subspace, even for a same database.

5.3. Comparison with the State of the Art

In this section, we compare the accuracy obtained using the proposed unsupervised $AL$-score with the results reached in the state of the art on the three considered sets. For a fair comparison, these sets have the same experimental protocol (number of classes, image size, number of images for each class, total number of images, and accuracy evaluation method), and only the works that apply a single color space strategy are mentioned. In addition to the nearest neighbor classifier, we propose also to use the SVM classifier during the classification stage of our approach since the best accuracy reached in the state of the art on Outex-TC-00013 and NewBarkTex with a single color space strategy has been reached thanks to this classifier. A one versus one SVM classifier with a linear kernel is here considered. The results are summarized in

Table 6. Comparison between the classification accuracies reached with the Outex-TC-00013 set.

Features	Color Space	Classifier	R (%)
3D-adaptive sum and difference histograms [9]	$ISH$	SVM	95.8
3D color histogram [43]	$HSV$	1-NN	95.4
Fractal descriptors [44]	$RGB$	LDA	95.0
EOCLBP with selection thanks to the $AL$-score	$RGB$	SVM	94.9
Haralick features [5]	$RGB$	5-NN	94.9
3D color histogram [45]	$RGB$	3-NN	94.7
3D color histogram [46]	I-$HLS$	1-NN	94.5
Haralick features [11]	$RGB$	1-NN	94.1
EOCLBP/C [47]	$HSV$	SVM	93.5
EOCLBP with selection thanks to the $AL$-score	$RGB$	1-NN	93.4
EOCLBP with selection thanks to the $ASL$-score [28]	$RGB$	1-NN	93.4
EOCLBP [27]	$RGB$	1-NN	92.9
Reduced Size Chromatic Co-occurrence Matrices [4]	$HLS$	1-NN	92.5
Between color component LBP histogram [43]	$RGB$	1-NN	92.5
Color histogram + LBP-based features [48]	$RGB$	1-NN	90.3
Wavelet coefficients [49]	$L^{*}{a}^{*}{b}^{*}$	BDC	89.7
Autoregressive models + 3D color histogram [46]	I-$HLS$	1-NN	88.9
Halftoning local derivative pattern + Color histogram [50]	$RGB$	1-NN	88.2
Autoregressive models [46]	$L^{*}{a}^{*}{b}^{*}$	1-NN	88.0
Within color component LBP histogram [43]	$RGB$	1-NN	87.8
Mixed color order LBP [51]	$RGB$	1-NN	87.1
Features from wavelet transform [52]	$RGB$	7-NN	85.2
Color contrast occurrence matrix [53]	$RGB$	1-NN	82.6
Fuzzy aura matrices [54]	$RGB$	1-NN	80.2

SVM: Support Vector Machine, LDA: Linear Discriminant Analysis, BDC: Bayes Decision Classifier.

,

Table 7. Comparison between the classification accuracies reached with the USPTex set.

Features	Color Space	Classifier	R (%)
Color histogram + LBP-based features [48]	$RGB$	1-NN	95.9
Local jet + LBP [55]	Luminance	LDA	94.3
Halftoning local derivative pattern + Color histogram [50]	$RGB$	1-NN	93.9
EOCLBP with selection thanks to the $AL$-score	$YUV$	1-NN	93.2
EOCLBP with selection thanks to the $AL$-score	$YUV$	SVM	87.9
Fractal descriptors [56]	Luminance	LDA	85.6
Mixed color order LBP [51]	$RGB$	1-NN	84.2

and

Table 8. Comparison between the classification accuracies reached with the NewBarktex set.

Features	Color space	Classifier	R (%)
Dominant and minor sum and difference histograms [57]	$RGB$	SVM	89.6
EOCLBP with selection thanks to the $AL$-score	$RGB$	SVM	84.9
Fine Texture and Coarse Color Features [58]	$HSV$	NSC	84.3
3D-adaptive sum and difference histograms [9]	$RGB$	SVM	82.1
EOCLBP with selection thanks to the $AL$-score	$RGB$	1-NN	81.4
EOCLBP with selection thanks to the $ICS$-score [27]	$RGB$	1-NN	81.4
EOCLBP with selection thanks to the $ASL$-score [28]	$RGB$	1-NN	81.4
Mixed color order LBP [51]	$RGB$	1-NN	77.7

NSC: Nearest Subspace Classifier.

.

From these tables, we can see that the second best accuracy result obtained on the Outex-TC-00013 set (95.4%) is achieved thanks to a simple 3D color histogram, although it only characterizes the color distribution within the $HSV$ color space, and does not take into account the spatial relationships between neighboring pixels, as a color texture feature should. This inconsistency is due to the fact that the Outex-TC-00013, as well as the USPTex sets, present a major drawback: The partitioning used to build these two sets consists in extracting the training and the testing subimages from a same original image. However, such a partitioning, when it is combined with a classifier such as the nearest neighbor classifier, leads to biased classification results [42]. Indeed, testing images are spatially close to training images. They are thus correlated and a simple 3D color histogram reaches a high classification accuracy [43]. For the NewBarktex set, the training and the testing subimages come from different original images to ensure that color texture images are less correlated as possible. The analysis of the results is thus more efficient and interpretable on this image set. The best accuracy rate (89.6%) is obtained thanks to the dominant and minor sum and difference histograms [57]. Selecting LBP histograms thanks to our proposed $AL$-score allows to get close to this highest rate, particularly when a SVM classifier is considered to classify the testing images. In this case, the classification accuracy reaches the promising result of 84.9%. This additional experiment highlights the merit of the unsupervised $AL$-score when it is associated with the SVM classifier.

6. Conclusions

We have proposed to adapt the traditional unsupervised feature selection scores in order to rank and select LBP histograms for color texture classification: The Adapted Variance ($AV$-score) and the Adapted Laplacian ($AL$-score) scores have thus been presented.

For each one of the nine LBP histograms extracted from a color texture, a score is assigned using one of the proposed adapted scores. The histograms are then ranked in order to select the most discriminant ones and thus build a low dimensional relevant subspace, in which a classifier operates.

Experiments on Outex-TC-00013, USPTex and NewBarkTex sets have shown the interest of performing a LBP histogram selection before classifying the different images. This selection improves the classification accuracy while reducing the dimension of the histogram subspace. The $AL$-score outperforms the $AV$-score and gives performances comparable or even better than the supervised $ASL$-score and $ICS$-score.

For future research directions, we propose to associate the $AL$-score with a multi color space approach [29]. Moreover, an additional experimentation can be realized in the short term perspective: Similarity can be derived from a given distance by kernelization (exponential with Euclidean distance in the conventional approach of Laplacian score). As the Jeffrey divergence can also be kernelized, it would be interesting to study the trend of the results considering a kernelized Jeffrey measure as similarity measure and, more generally, the impact of the distance and similarity measure on the classification performances.