Adaptive Marginal Median Filter for Colour Images

Abstract

This paper describes a new filter for impulse noise reduction in colour images which is aimed at improving the noise reduction capability of the classical vector median filter. The filter is inspired by the application of a vector marginal median filtering process over a selected group of pixels in each filtering window. This selection, which is based on the vector median, along with the application of the marginal median operation constitutes an adaptive process that leads to a more robust filter design. Also, the proposed method is able to process colour images without introducing colour artifacts. Experimental results show that the images filtered with the proposed method contain less noisy pixels than those obtained through the vector median filter.

1. Introduction

Noise is often introduced into digital images during the acquisition and transmission processes because of different reasons such as CCD sensor malfunction, transmission errors, storage faults, and difficult acquisition conditions. The presence of noise hampers the automatic processing of digital images and also affects their visualization quality. This implies that the noise reduction task, also known as image filtering, is a fundamental step in any computer vision system. In this context, several types of noise have been studied. Here we focus on the impulse noise case, which affects a portion of the image pixels, replacing their original values with other very different ones.

The earliest filters were developed to process gray-scale images and were based on linear approaches. However, it was found that nonlinear methods exhibit better performance and, in particular, that the median operator is the most robust method when the images are contaminated with impulse noise [1].

Recently, the interest in employing colour images has grown in a wide range of applications, which has motivated the development of colour image filters. The simplest attempts for processing colour images are based on applying a method for gray-scale images in each of the three colour channels independently [2]. However, it is well known that this way of processing is not appropriate for colour images because there exists a high correlation among the colour image channels which is not considered by this kind of methods [2,3]. For instance, the Vector Marginal Median Filter (VMMF) [2] uses the scalar median operator in each of the colour channels independently to obtain the filtered image. However, when obtaining the colour for a particular image pixel, the VMMF may combine components of image pixels that may be very different, which in turn may generate artificial colours (also known as artifacts). This implies that the VMMF is not useful for real applications since it cannot adapt its performance to the existence of correlation among the image channels.

From another point of view, the Vector Median Filter (VMF) [4] proposes to process the colour images by treating them as a vector field in order to take into account the interchannel correlation. The family of vector filters inspired by the VMF, which includes the Directional Vector Filter among others [3,5], is based on the theory of robust statistics [1,6]. The filters of this family, and specially the VMF, can perform quite robustly in impulse noise reduction without introducing colour artifacts, since they appropriately consider the colour components correlation. These filters select the vector that is closest to the rest of the vectors in a given population based on distance measure to be the output. However, the noise reduction capability of these filters is lower than the VMMF. For instance, in the case of a vector population where all vectors have one noisy component, the VMF will always select a noisy vector as output, since it cannot adapt to this extremely noisy situation.

In this work we propose a method to improve the performance of the VMF in this sense. Our method is based on applying a VMMF process over a group of pixels selected using the Vector Median (VM) of the population. As a result, a more robust adaptive filter design able to process colour images without introducing colour artifacts is obtained. As we will show in the experimental section, the images obtained through the proposed method contain less noisy pixels than those obtained through the VMF. The proposed method is intended to be used within more complex filtering procedures, for instance, in the noise reduction step, where the VMF is frequently used [7–9].

The paper is organized as follows. Section 2 recalls the basics of the VMMF and the VMF. The proposed filtering process is introduced in Section 3. Experimental results and comparisons are provided in Section 4 and, finally, Section 5 presents the conclusions.

2. Vector Marginal Median Filter and Vector Median Filter

Denote by F a colour (or multichannel) image to be processed and let W be a filtering window centered on the pixel under processing of size N × N, N = 3, 5, 7 … containing N² = n pixels. The colour vectors in W are denoted as ${\mathbf{F}}_j=(F_j^R, F_j^G, F_j^B)$, j = 0, 1, …, n − 1, as usual in the RGB colour space. The distance between two vectors F_i, F_j is denoted as ρ(F_i, F_j). In this work, we take the Euclidean distance as the ρ function, but any other distance metric could be used instead [2,3,10–13].

2.1. Vector Marginal Median Filter

The output of the VMMF when processing the center pixel of W is the vector,

$$\mathbf{VMM}=\left(\mathit{med}\left(\left\{F_0^R,\dots , F_{n-1}^R\right\}\right), \mathit{med}\left(\left\{F_0^G,\dots , F_{n-1}^G\right\}\right), \mathit{med}\left(\left\{F_0^B,\dots , F_{n-1}^B\right\}\right)\right)$$

(1)

where med denotes the statistical median operation. In this way, no relation among the colour components of VMM is considered, which leads to the problems mentioned in the previous section. On the other hand, VMM contains the most robust estimation in each component, which implies that its noise reduction capability is the highest.

2.2. Vector Median Filter

The VMF approaches the problem of noise reduction by looking for the most robust vector in the population. For this, each vector in the filtering window is associated with an accumulated distance to all other vectors which is computed as $R_i={\sum }_{j=0}^{n-1}\rho ({\mathbf{F}}_i, {\mathbf{F}}_j)$. Thus, R_i is the distance associated to the vector F_i. Then, the colour vectors are ordered according to R_i, so that the ordering of the R_i’s: R₍₀₎ ≤ R₍₁₎ ≤ … ≤ R_(n−1), implies the same ordering of the vectors F_i’s: F₍₀₎ ≤ F₍₁₎ ≤ … ≤ F_(n−1). Given this order, the output VM = F₍₀₎, which is the colour vector associated to the minimum accumulated distance. Notice that because of the vector approach, the correlation among the VM components is considered, which avoids the generation of colour artifacts. However, in a very noisy context where all colour vectors contain some noisy component, VM will be noisy. In the following section we introduce a method intended to increase the noise reduction capability of the VMF in this sense.

3. Proposed Method: Adaptive Vector Marginal Median Filter

As mentioned above, the proposed method is based on the application of a VMMF operation over a selected group of colour vectors. To make this selection, we employ the VM of the filtering window W as follows.

Following the notation in Section 2, we take the vector median VM and we order the vectors in W according to their similarity with VM. So, we define the ordered set:

$$W^{\prime }=\left\{{\mathbf{F}}_{\left[0\right]}, {\mathbf{F}}_{\left[1\right]},\dots , {\mathbf{F}}_{\left[n-1\right]}\right\}$$

(2)

such that

$$\rho (\mathbf{VM}, {\mathbf{F}}_{\left[0\right]})\le \rho (\mathbf{VM}, {\mathbf{F}}_{\left[1\right]})\le \dots \le \rho (\mathbf{VM}, {\mathbf{F}}_{\left[n-1\right]})$$

(3)

where, obviously, F_[0] = VM.

We select a set of colour vectors, say S, which will be constituted by the m colour vectors most similar to VM. The objective of this selection is to adapt the performance of the method to the existence of correlation among the colour image channels. Notice that m is an adaptive parameter that relates VM with the number of pixels in W which are similar to it, which in turn are the colour vectors with a similar relation among their components. In this way,

$$S=\left\{{\mathbf{F}}_{\left[0\right]}, {\mathbf{F}}_{\left[1\right]},\dots , {\mathbf{F}}_{\left[m-1\right]}\right\}$$

(4)

where m ≤ n. If m is low enough m << n, S contains the VM and m − 1 colour vectors with similar components. Therefore, S contains a set of robust colour vectors, for they are similar to the VM and, moreover, all the vectors in S have similar components. In an optimal situation, m should be set to the number of colour vectors in W similar to VM. However, in a very noisy context, some noisy components will be also included in S. Now, we apply the VMMF operation over the colour vectors in S to achieve high noise reduction so that the method can perform well in very noisy situations. The proposed method output is obtained as follows:

$$\mathbf{AMM}=\left(\mathit{med}\left(\left\{F_{\left[0\right]}^R,\dots , F_{\left[m-1\right]}^R\right\}\right), \mathit{med}\left(\left\{F_{\left[0\right]}^G,\dots , F_{\left[m-1\right]}^G\right\}\right), \mathit{med}\left(\left\{F_{\left[0\right]}^B,\dots , F_{\left[m-1\right]}^B\right\}\right)\right)$$

(5)

In this way, we obtain a more robust vector than the VM because of the robustness of the median operation. Notice that now, given that the vectors in S are similar among them, which in turn implies that they have similar components, no colour artifacts will be generated. This happens because through the colour vector selection process the marginal median operation has been adapted to the similarities observed in the population, which overcomes the main drawback of the VMMF.

4. Experimental Results and Assessment

Impulsive noise corruption process affects only some pixels in the image while leaving other pixels unchanged. Typically, the noise process changes one or more colour components of the affected pixel by replacing its original values with other values which usually significantly deviates from the originals. The kind of noise which is the most difficult to detect and remove assumes that the impulse is a random uniformly distributed value within the signal range. For RGB images, we consider that the noise is independently introduced in each of the three colour channels with probability p, which means that the (3p − 3p² + p³) 100% image pixels are corrupted with noise. This noise model has been used to contaminate the test images in Figure 1.

The contaminated images have been filtered using the VMMF, VMF and the proposed AMMF. In the three cases we have used a 3 × 3 filtering window and we have filtered each image only once to better appreciate the performance differences. However, when the noise is high, several filtering processes are frequently necessary to obtain a totally clean image. Another possibility is to use a larger window size. For instance, it is known that using a 5 × 5 filtering window provides a higher noise reduction capability and could be more appropriate in general for filtering very noisy images. However, the method we propose is intended to be used within more complex filter designs in the noise reduction step, and most high performance advanced filters use a 3 × 3 filtering window [7–9]. Therefore, we consider that it is more interesting to study the performance of our method in this case.

The adaptive parameter m of AMMF should be set, in an optimal ideal situation, to the number of vectors similar to the vector median in each filtering window. Since we focus on extremely noisy situations, we expect this number to be quite low. Also, according to the filter design, as m increases, the performance of the proposed methods become more similar to the VMMF since more colour vectors are involved in the marginal median operation. This means that increasing m will increase both the noise reduction capability of the method and the likelihood to introduce colour artifacts in the output image (as we will see, numerical results in

Table 1. Performance comparison in terms of MAE, PSNR, and NCD when filtering the Boats image contaminated with random-value impulse noise with probability p in each colour channel.

Filter	p = 0.10				p = 0.20				p = 0.30				p = 0.40
	MAE	PSNR	NCD	NP%	MAE	PSNR	NCD	NP%	MAE	PSNR	NCD	NP%	MAE	PSNR	NCD	NP%
None	7.55	18.82	12.02	22.2	15.01	15.83	22.60	40.5	22.70	14.03	32.28	57.9	30.26	12.77	40.40	73.2
VMMF	4.30	29.99	3.74	0.9	5.24	28.07	5.06	2.1	6.78	25.55	7.31	3.7	9.52	22.56	11.13	6.5
VMF	4.67	29.51	3.19	0.7	6.18	26.79	4.72	2.0	8.82	23.44	8.29	5.2	12.93	20.32	13.76	10.2
AMMF3	4.73	29.09	3.28	0.7	6.01	26.53	4.58	1.8	8.18	23.24	7.75	4.8	11.86	20.08	13.13	9.8
AMMF4	4.89	29.13	3.33	0.6	5.97	26.88	4.55	1.5	7.99	23.68	7.60	3.9	11.59	20.47	12.86	8.3
AMMF5	4.77	29.02	3.41	0.9	5.68	27.03	4.63	1.9	7.43	23.98	7.37	4.6	10.73	20.79	12.30	9.1

–

Table 4. Performance comparison in terms of MAE, PSNR, and NCD when filtering the Flower image contaminated with random-value impulse noise with probability p in each colour channel.

Filter	p = 0.10				p = 0.20				p = 0.30				p = 0.40
	MAE	PSNR	NCD	NP%	MAE	PSNR	NCD	NP%	MAE	PSNR	NCD	NP%	MAE	PSNR	NCD	NP%
None	7.46	18.82	11.57	22.8	14.70	15.88	21.63	41.4	22.33	14.08	30.87	58.8	29.70	12.86	38.77	73.5
VMMF	5.00	28.63	3.89	1.2	6.25	26.81	5.72	2.0	8.06	24.56	7.90	3.5	10.87	22.05	11.00	5.9
VMF	5.42	28.17	2.97	1.0	7.46	25.63	4.54	1.9	10.19	22.84	7.71	5.4	14.20	20.14	12.49	10.0
AMMF3	5.56	27.76	3.14	1.0	7.38	25.38	4.65	1.9	9.53	22.73	7.61	5.0	13.10	19.98	12.51	9.5
AMMF4	5.87	27.85	3.32	0.8	7.32	25.77	4.70	1.4	9.26	23.20	7.58	4.1	12.72	20.33	12.34	7.9
AMMF5	5.73	27.77	3.44	1.1	6.86	25.95	4.88	1.9	8.62	23.47	7.65	4.6	11.86	20.64	12.07	8.8

support this reasoning). Given that our objective is to improve the noise reduction capability of the VMF but avoiding as much as possible the introduction of colour artifacts, we have set m to the minimum value that makes sense to use: m = 3.

In addition to visual comparison, to assess the quality of the different filters we have used the objective quality measures Mean Absolute Error (MAE), Peak Signal to Noise Ratio (PSNR), and Normalized colour Difference (NCD) defined as follows [2,3]:

$$\mathit{MAE}=\frac{\sum _{i=1}^{N\cdot M}\sum _{q=1}^Q\left|F_i^q-{\widehat{F}}_i^q\right|}{N\cdot M\cdot Q}$$

(6)

$$\mathit{PSNR}=20 \text{log} \left(\frac{255}{\sqrt{\frac{1}{\mathit{NMQ}}\sum _{i=1}^{N\cdot M}\sum _{q=1}^Q{\left(F_i^q-{\widehat{F}}_i^q\right)}^2}}\right)$$

(7)

where M, N are the image dimensions, Q is the number of channels of the image (Q = 3 for colour images), and $F_i^q$ and ${\widehat{F}}_i^q$ denote the q^th component of the original image vector and the filtered image, at pixel position i, respectively, and

$${\mathit{NCD}}_{\mathit{Lab}}=\frac{\sum _{i=1}^{N\cdot M}{\Delta E}_{\mathit{Lab}}}{\sum _{i=1}^{N\cdot M}{E}_{\mathit{Lab}}^{*}}$$

(8)

where ${\Delta E}_{\mathit{Lab}}={\left[{\left(\Delta L*\right)}^2+{\left(\Delta a*\right)}^2+{\left(\Delta b*\right)}^2\right]}^{\frac{1}{2}}$ denotes the perceptual colour error and $E_{\mathit{Lab}}^{*}={\left[{\left(L*\right)}^2+{\left(a*\right)}^2+{\left(b*\right)}^2\right]}^{\frac{1}{2}}$ is the norm or magnitude of the original image colour vector in the L*a*b* colour space.

Also, to assess the robustness of the methods we have computed the percentage of noisy pixels (NP%) in the output images using the simple but effective method proposed in [14,15]. This detection method considers a colour pixel as noisy if it has less than m colour pixels within Euclidean distance lower than d in a 3 × 3 neighborhood. In particular we have set m = 2 and d = 35.

Figures 2–5 show some noisy images and the respective filtering results using the VMMF, VMF and AMMF (m = 3). We can see that, overall, the highest noise reduction ability is exhibited by the VMMF. However, this filter also introduces many colour artifacts near edges (see for instance dark zones edges in Figure 2(b)). On the other hand, the images filtered with the AMMF contain less noisy pixels than those filtered with the VMF and no colour artifacts are introduced by any of these two methods. This fact is also illustrated in Tables 1–4 where we can see the NP for different images after filtering with the VMMF, VMF and AMMF_m varying the value of m in {3, 4, 5}. In these tables AMMF obtains NP values lower than VMF. Also, we can see that when the noise is low (up to p = 0.20, which implies about 40% of noisy pixels) the images filtered with VMMF have higher NP than the AMMF and VMF. This happens because although VMMF has the higher ability to suppress noise it also generates colour artifacts, which increases NP in the output images. This fact implies that VMMF is not reliable and should not be used in practical applications. When the noise is high (p ≥ 0.30, which implies more than 50% of noisy pixels), VMMF still introduces artifacts but NP is lower because the percentage of noise reduced is much larger than the artifacts introduced. With respect to the rest of the quality measures, i.e., MAE, PSNR and NCD, the best results are obtained by VMMF, but notice that these measures do not consider specifically the introduction of colour artifacts. When the noise is high, AMMF outperforms VMF in terms of MAE and NCD. From the visual results and the NP, it would be logical that AMMF always outperforms VMF, but these measures do not perfectly match with these criteria in such a noisy context. Finally, if we analyze the performance of AMMF for the different values of m considered, we can see that the best numerical results are obtained for m = 4. However, increasing the value of m also increases the likelihood to introduce colour artifacts, so we prefer to set m = 3. Most probably, m = 4 achieves the best trade-off between noise suppression and artifact generation, which implies better numerical values, but since it is a primary objective to avoid the generation of artifacts, we set m = 3. These results allow us to conclude that AMMF (m = 3) is the method able to obtain the most robust and reliable results since its ability to suppress noise is higher than VMF and it does not generate colour artifacts, which is an advantage with respect to VMMF.

5. Conclusions

In this paper we have presented an adaptive method for impulse noise reduction in colour images whose objective is to improve the noise reduction capability of the classical vector median filter. The filter is based on the selection of a few vectors in a population using the vector median and the application of a vector marginal median filtering over the selected vectors. The robustness and reliability of the method is achieved because the selection of vectors adapts the performance of the marginal median operation to an appropriate context. Experimental results show that the images filtered with the proposed method contain less noisy pixels than those obtained through the vector median filter. Also, the proposed method is able to process colour images without introducing colour artifacts, which is an advantage with respect to the vector marginal median filter. These results suggest that a more robust correlated filtering method might be obtained, which opens an interesting line of research. The proposed method can be used within more complex filtering procedures, for instance, in the noise reduction step, where the VMF is frequently used.