Detection of Multi-Pixel Low Contrast Object on a Real Sea Surface ^†

Abstract

Video images captured at long range often show low-contrast floating objects of interest on a sea surface. A comparative experimental study of the statistical characteristics of reflections from floating objects and from the agitated sea surface showed differences in the correlation and spectral characteristics of these reflections. The functioning of the recently proposed modified matched subspace detector (MMSD) is based on the separation of the observed data spectrum on two subspaces: relatively low and relatively high frequencies. In the literature, the MMSD performance has been evaluated in general and using only a sea model (i.e., additive Gaussian background clutter). This paper extends the performance evaluating methodology for low contrast object detection using only a real sea dataset. The methodology assumes an object of low contrast if the mean and variance of the object and the surrounding background are the same. The paper assumes that the energy spectrum of the object and the sea are different. The paper investigates a scenario in which an artificially created model of a floating object with specified statistical parameters is placed on the surface of a real sea image. The paper compares the efficiency of the classical matched subspace detector (MSD) and MMSD for detecting low-contrast objects on the sea surface. The article analyzes the dependence of the detection probability at a fixed false alarm probability on the difference between the statistical means and variances of a floating object and the surrounding sea.

1. Introduction

We tackle the problem of multi-pixel floating objects detection in image sequences, which arises in search and track video systems [1,2,3,4,5,6,7,8,9,10,11]. In many instances, the objects are small and low contrast in a surrounding background environment. The known methods, such as clutter removing [1,2,3], subspace projection [6,7,8], the entropy-based method [11], and multi-frame, solve the detection in a fluctuating background. The modified mean subtraction filter (MMSF) [3] improved the mean subtraction filter (MSF) but only for a high signal-to-background ratio (SBR). Conventional low contrast target detection methods, such as median filter [3], least mean square filter [2,12], and morphology-based methods [10], are used to reduce only the background clutter. Application of such methods to problems of target detection show limited detection performance enhancement since these methods use fixed filters.

Detection algorithms based on statistical methods are widely used in remote sensing systems where channel noise and random intense background environment are present [13,14,15,16,17,18,19,20,21,22,23,24,25]. The well-known matched and matched subspace filters [15] are such algorithms. Adaptive algorithms and algorithms that take into account the specific features of scenarios, based on the generalized likelihood ratio test (GLRT), were considered in [15,16,21,22,23]. A modified adaptive subspace detector (MASD) was proposed for detecting faintly discernible objects during image processing [17,18,19,20]. The problem of detecting low-contrast objects on an agitated sea surface was proposed in [25].

In this paper, we continued to study the possibility of the detection of small low contrast fluctuating objects on the agitated sea surface in the presence of channel noise. As shown in [19], the detection algorithm obtained by the GLRT method depends on a priori information about the parameters of the object. In contrast to [25], two algorithms obtained in [19] are considered here: the first (MSD) was obtained in the absence of a priori information about the object size, the second algorithm (MMSD) was obtained provided that the size of the sub-image is much smaller than the object size.

Algorithms for detecting objects on the surface of the sea should be slightly sensitive to sensor movement and water spray. The common drawbacks of the published papers are the assumption that the background and channel noise are almost Gaussian processes and the lack of a low contrast target model. To eliminate these shortcomings, this study uses images of real sea surfaces in various weather conditions and determines the contrast of a floating object as the difference between the statistical averages and the variances of the compared surfaces. In this study, the low contrast floating object has a statistical mean and variance almost equal to the sea surface. This paper investigates the dependence of the detection probability on the difference between the mean and variance of the floating object and the surrounding sea surface. For such a study, it is necessary to change the values of these differences. It is very difficult to experimentally obtain real images of floating objects with different mean values and variances. Therefore, this study uses a model of reflections from a floating object, in which both average values and variances can be changed.

This approach allows change in the difference between the statistical parameters of the object and the environment. The aim of this paper is to study the performance of two detectors (MSD and MMSD) in the case of low-contrast floating objects detection.

2. Floating Object Signal Model

In this paper, using the experimental data, we investigate the relationship between two techniques for the detection of small low contrast floating objects on the sea surface in image sequences: these are the well-known MSD and recently proposed MMSD techniques [18]. Unlike previous studies, this study uses real images of the sea surface on which a model of a floating object is placed. As is well known, the creation of a model of an agitated sea surface creates significant difficulties.

Therefore, we do not use a model of reflections from the sea surface in this study, replacing it with real images of the agitated sea surface under various (three types) weather conditions. We use a generic model of reflections from a floating object [18]. This approach allows, firstly, avoidance of possible errors due to the inaccuracy of the model of reflections from the sea and, secondly, introduction of a model of a floating object, to control the magnitude of the contrast of the floating object relative to the sea surface.

This approach allows a set to be selected for contrast analysis of the object in relation to the sea surface. Contrast is usually defined as the difference between the target average intensity and the average background intensity. What needs to be considered is that the definition of contrast given above only applies to the local area of an image instead of the entire image.

In this paper, we expand the definition of contrast by introducing as a contrast parameter the difference between the variances of the object and the sea. The detection of a floating object under the condition of minimum contrast (i.e., where the difference between the means and the difference between the variances of the object and the sea is close to zero) depends on the difference in the covariance matrices of reflections from the object and the sea (or from their energy spectra). Therefore, the detector must be sensitive to the difference between the covariance matrices of the object and the sea. This paper analyzes this sensitivity for two detectors.

This section discusses a model of a floating object on a rough sea surface. First, the size of the rectangular sub-image (K × M) is selected, in which processing will be performed in order to detect a signal from a floating object into this sub-image. Then, the analyzed image is divided into a number of such non-overlapping sub-images. We assume that the sub-image (K × M) is significantly smaller than the dimensions of the floating object (N × L). The object signal is modeled as a two-dimensional matrix (N × L).

The floating object model is a matrix, each column of which is formed as a vector of different shapes. A linear model with a Vandermonde matrix is used to form each column. The study assumes the detection of a solid floating object. Therefore, the floating object model assumes a change in the maximum frequency of each column, but this frequency is always less than the maximum frequency of the signal reflected from the sea surface. The r-th column-vector of the object sub-image model is represented by:

$$s_r=\mathrm{H}{\theta }_r$$

(1)

where r = 1, 2, …, L; H is the object mode matrix (Vandermonde matrix) with discrete complex exponential elements:

$$\mathrm{H}=\left[\begin{array}{cc}\begin{array}{c}1\\ z_0\\ ⋮\\ z_0^{N-1}\end{array}& \begin{array}{c}1\\ z_1\\ ⋮\\ z_1^{N-1}\end{array}\end{array} \begin{array}{cc}\begin{array}{c}\cdots \\ \cdots \\ \cdots \\ \cdots \end{array}& \begin{array}{c}1\\ z_p\\ ⋮\\ z_p^{N-1}\end{array}\end{array}\right]$$

(2)

where $z_n^i=h_{in}=\frac{1}{\sqrt{N}}\exp \left(\frac{j2\pi in}{N}\right)$, the subscripts n and i denote the column and row number of the matrix H, respectively; i = 0, 1, 2, …, N − 1; n = 0, 1, 2, …, p; j = √(−1). The linear model (1) can be interpreted as a linear combination of columns (complex harmonics) of H, weighted by their respective coefficients (amplitudes) θ_r = [θ_1r, θ_2r, …, θ_pr]. The parameter θ_r is a priori unknown and in this paper is a sample from a random uniform distribution.

Thus, the multipixel model of a floating object is represented by a matrix of size (N × L) consisting of L vectors s_r.

Examples of object models with different mean values and variances using the eight first Fourier harmonics are shown in Figure 1. Comparisons of the object model with the same mean and variance values, but using the first three Fourier harmonics, are shown in Figure 2.

3. Problem Statement and Detector Design

The purpose of this section is a statistical description of the detection problem and the corresponding suboptimal algorithms synthesized by the method GLRT. The problem is to test the statistical hypothesis $H_0$ against the alternative $H_1$ and to select the one that is most consistent with the prior models and the observations. We will consider two possible scenarios for detecting a floating object on the sea surface, using one frame and a sub-image (K × M), in which the detection is carried out.

The first scenario assumes that there is no information about the size of the floating object. Therefore, it is generally assumed that the detection sub-image can be larger than the size of the object (K > N, M > L). Then, under hypothesis $H_1$, the received signal consists of the sum of the signals reflected from the object, the sea and the channel noise. The first scenario can be described as follows:

$$\left\{\begin{array}{c}{H}_0: x_m=c_m+n_m, m\in \left[1,M\right],\\ H_1: \left\{\begin{array}{c}{x}_i=s_i+n_i, i\in \left[1,L\right],\\ x_j=c_j+n_j, j\in \left[L+1,M\right],\end{array}\right.\end{array}\right.$$

(3)

The hypothesis $H_0$ assumes that the received signal vector is the sum of the signal reflected from the sea surface $c_m={\left[c_{1m } c_{2m }\cdots c_{Km}\right]}^T$ and the additive channel noise $n_m={\left[n_{1m } n_{2m }\cdots n_{Km}\right]}^T$. Under an alternative hypothesis, the received signal consists of signals reflected from the object $s_i={\left[s_{1i } s_{2i }\cdots s_{Ni}\right]}^T$, the sea $c_j={\left[c_{1j } c_{2j }\cdots c_{Kj}\right]}^T$ and additive normal noise $n_i={\left[n_{1i } n_{2i }\cdots n_{Ni}\right]}^T$, $n_j={\left[n_{1j } n_{2j }\cdots n_{Kj}\right]}^T$. The article [19] synthesizes the detection algorithm for this scenario. For simplicity, it is assumed that the signal reflected from the sea surface can be represented as a normal random uncorrelated process $c_m~N\left(0,{\sigma }_c^{2}I\right)$ and channel noise $n_m~N\left(0,{\sigma }_n^{2}I\right)$, where ${\sigma }_c^2>{\sigma }_n^2.$ This algorithm was obtained at unknown object size and known spectral frequency range of the reflected signal. To describe the detection algorithm, the article [19] used the orthogonal (N × N) projection matrix $P=\mathrm{H}{\left({\mathrm{H}}^H\mathrm{H}\right)}^{-1}{\mathrm{H}}^H$.

The resulting algorithm practically coincides with the well-known MSD algorithm:

$$T_{MSD}={\sum }_{m=1}^M\{\frac{1}{{\sigma }_{c+n}^2}{x}_m^T{P}_s{x}_m-q{x}_m^T{P}_s^{\perp }{x}_m\}{\frac{>}{<}}_{H_0}^{H_1}\eta ,$$

(4)

where η is a threshold that determines the probability of a false alarm, q = ${\sigma }_n^{-2}-{\sigma }_{c+n}^{-2}$, $P_s$ is K$\times$ K orthogonal projection matrix onto the object subspace 〈H〉, $P_s^{\perp }=I-P_s$.

The second scenario assumes that the object is larger than the sub-image so that there is always one sub-image that is completely covered by the object. In this case, we can assume that N = K and L = M.

The null statistical hypothesis corresponds to the case of observing the sea surface without an object, and the alternative hypothesis corresponds to the case of the presence of an object on the sea surface. In this case we can represent two statistical hypotheses $H_0$ and $H_1$:

$$\left\{ \begin{array}{c}{H}_0:x_m=c_m+n_m\\ H_1:x_m=s_m+n_m,\end{array}\right. m\in \left[1,M\right]$$

(5)

It is known that the Vandermonde matrix (2) consists of columns orthogonal to each other. The paper assumes that columns from 1 to p describe reflections from a floating object, and columns from p + 1 to K describe an orthogonal subspace 〈${\mathrm{H}}^{\perp }$〉, in which the energy of signals reflected from the sea surface is present and there is no signal from the object. This assumption is consistent with the experimental results of studying the spectral composition of reflections from the sea and from the object. To describe the detection algorithm, the article [19] used the orthogonal projection matrix $P_{\perp }$= I − P onto the orthogonal subspace 〈${\mathrm{H}}^{\perp }$〉, where I is an identity matrix (K × K). The MMSD [19] is:

$${T_{MMSD}}_{ }={{\displaystyle \sum }}_{m=1}^M\left[\frac{x_m^T{x}_m}{K{\sigma }_0^2}-b · ln\frac{x_m^T{P}_{\perp }{x}_m}{K{\sigma }_0^2}\right] \begin{array}{c}\begin{array}{c}{H}_1\\ \ge \end{array}\\ \begin{array}{c}<\\ H_0\end{array}\end{array}{\eta }_1,$$

(6)

where ${\sigma }_0^2={\sigma }_c^2+{\sigma }_n^2$ is the background plus noise variance of the pixel, b is sensitive factor, ${\eta }_1$ is the threshold.

4. Comparative Performance Assessment

In this section, the performance of the MMSD and MSD in terms of detection probability (P_d) at a fixed false alarm probability P_fa = 10⁻³ is assessed. The paper considers the case of a low-contrast object on the sea surface. A low-contrast object is understood as an object whose statistical mean and variance are close to the corresponding values of reflections from the sea surface. The paper uses real videos of the sea surface under various weather conditions. Four different videos were selected, which are presented in Figure 3.

Figure 3a–c shows images of sea surfaces with real low-contrast floating objects, and Figure 3d shows a real sea surface with a low-contrast model of a floating object. All the following figures show the curves averaged over these four sea surface types. It is known that the MSD calculates the power of the input signal in a bandwidth where the reflected energy from the object is significant. The signal power depends on the statistical mean and variance of the signal. Therefore, the MSD cannot detect signals in the background noise if the signal and noise power are the same in a given bandwidth.

MMSD, unlike MSD, is also sensitive to power in the orthogonal subspace, in which signals from the sea and channel noise are present at the null hypothesis, or only channel noise is present at the alternative hypothesis.

Figure 3 shows the various surfaces of an agitated sea with poorly visible real objects. In these images, the differences between means and variances of the object signal and sea signal do not exceed 10%.

This study required the presence of floating objects with different intensities of reflections (both statistical averages and variances). Therefore, a technique was used to place an artificially created model of a floating object on a real image of the sea. The statistical mean and variance of the model were changed programmatically.

Typical averaged spectra of reflections from a typical unobtrusive floating object and a typical agitated sea surface are shown in Figure 4a,b. The width of the spectrum of reflections from a floating object was less than 10 Hz (0–10), and from the rough sea surface it reached 25 Hz (0–25). Using these experimental data, the parameter p was selected in the detection algorithms.

To obtain the detection characteristics, a site was selected on the sea surface at a distance of approximately 200 m from the camera, with a 5 min video produced. Each frame was divided into windows (10 × 10), which covered the entire surface of each frame. Each window was processed in order to detect a floating object in it. The processing results for each window were compared with the set threshold. If the threshold was exceeded, a decision was made on the presence of a floating object inside the analyzed window. The dimensions of the floating object significantly exceeded the size of the window where processing and detection were carried out (10 × 10); the average power of the floating object reflections and the standard deviation changed during the experiments. Additive channel noise with a given intensity was added to all images. To implement the detection, we first selected the window of the sea image without an object in which the detections were carried out in order to establish a threshold providing a false alarm probability of 0.001. In the selected sub-image, the detection process was implemented 10,000 times in 10,000 frames. The output values of the detectors (10,000 numbers) were analyzed and ordered in ascending order.

In this paper, the problem of detecting a low-contrast floating object was solved. For this, the statistical means and variances of the object model were changed in the course of the experiments, and the dependence of the detection probability on the difference between the average object and the sea was analyzed for different values of the difference between the variances of the object and the sea. The ratio of the channel noise power to the sea power also changed.

In Figure 5 the characteristics of the MSD are given. In Figure 5a,b the dependences of the detection probability on the difference between the average object and the sea are given for different ratios of the channel noise power to the power of reflections from the sea NBR (noise-to-background ratio) and different coefficient v = OP/BP (object power-to-background power).

Analysis of the curves shows that the detection quality of the MSD is highly dependent on the difference between the average signals reflected from the object and from the sea. At the same time, a significant increase in the channel noise power insignificantly worsens the detection quality. An increase in the difference between the dispersions of reflections from the object and from the sea increases the detection quality significantly. However, the MSD does not provide high detection quality with a small difference between the average reflections from the object and from the sea.

In Figure 6 and Figure 7 the characteristics of the MMSD are given. Comparison of the graphs in Figure 6a,b shows that the quality of the MMSD strongly depends on the channel noise power. With an increase in the channel noise power, the probability of detecting MMSD deteriorates.

However, with a small ratio of the channel noise power to the power of reflections from the sea, the MMSD is able to detect a low-contrast object even with a zero difference between the average signals reflected from the object and from the sea.

As with the MSD, an increase in the difference between the dispersions of the reflections from the object and from the sea increases the detection quality.

Comparison of the graphs in Figure 7a shows that the quality of the MMSD strongly depends on the sensitivity coefficient b, which determines the contribution of the second term in the MMSD algorithm. Figure 7b suggests that, with an increase in the sensitivity coefficient, the dependence of the detection quality on the power of the channel noise increases. From the graphs produced, it follows that to detect low-contrast floating objects, it is advisable to use the MMSD algorithm, which has a high detection quality even with equal values of the mean and dispersion of signals reflected from the object and from the sea.

To explain this, attention should be paid to the second term in the MMSD, which increases with an increase in the difference between the power of the signals reflected from the sea and the channel noise. This algorithm assumes that the MMSD detector is able to estimate the power of noise or power of reflections from the sea in the frequency range in which there is no signal reflected from the object. This is valid for floating solid objects. Figure 4 shows the plots of the Fourier spectra modules characteristic of reflections from floating solid objects and from an agitated sea.

5. Conclusions

The main results obtained in the paper:

The experiments carried out show that, provided that reflections from a floating object and from the sea surface have the same statistical averages and variances (low-contrast object), the use of the classical detection algorithm MSD does not achieve high detection quality.

The experimental results show that the spectral energies of reflections from a rough sea surface and from a floating object are different in the upper frequency range of the received signals.

MMSD allows realizing high quality detection of a low-contrast floating solid object, because it is sensitive to spectrum energy in different frequency ranges (e.g., the high frequency range and the low frequency range).

The obtained results are confirmed by experimental data under various sea surface scenarios.

Basic requirements when using MMSD:

To use MMSD, a priori information is required on the width of the spectrum of signals reflected from a floating object.

The size of the window in which the object is detected must be smaller than the size of the object, i.e., a priori information about the minimum dimensions of the object is required.

Possible directions for further research:

Synthesis of a suboptimal detector based on GLRT under the condition of division of the frequency range into three or more frequency subspaces. Recognition of low-contrast objects located on a surface with random parameters can be used in various technologies, for example, for the automated recognition of anomalies in objects.