Infrared Target-Background Separation Based on Weighted Nuclear Norm Minimization and Robust Principal Component Analysis

Abstract

The target detection ability of an infrared small target detection (ISTD) system is advantageous in many applications. The highly varied nature of the background image and small target characteristics make the detection process extremely difficult. To address this issue, this study proposes an infrared patch model system using non-convex (IPNCWNNM) weighted nuclear norm minimization (WNNM) and robust principal component analysis (RPCA). As observed in the most advanced methods of infrared patch images (IPI), the edges, sometimes in a crowded background, can be detected as targets due to the extreme shrinking of singular values (SV). Therefore, a non-convex WNNM and RPCA have been utilized in this paper, where varying weights are assigned to the SV rather than the same weights for all SV in the existing nuclear norm minimization (NNM) of IPI-based methods. The alternate direction method of multiplier (ADMM) is also employed in the mathematical evaluation of the proposed work. The observed evaluations demonstrated that in terms of background suppression and target detection proficiency, the suggested technique performed better than the cited baseline methods.

1. Introduction

The various target detection applications that exist today are early warning, infrared search and tracking systems (IRST), and medical imaging; all rely heavily on the capability of ISTD. Due to the poor SCR, it is challenging to compute imaging distance, size, and texture [1,2,3]. Many researchers have performed exceptionally well in the past. So far, algorithms have been in two categories: sequential and single-frame detection [4]. The first category makes use of both spatial and temporal information, which is not easily available in real-time applications, and therefore, performance is lacking for methods based on such approaches. On the other hand, the detection algorithm must be extremely fast and accurate in locating the target.

As a result, the single-frame detection approach plays an important role and is heavily used. The motivation for the proposed approach is the existence of weakness in the existing IPI models. Although the infrared patch-image (IPI) model has proven successful in target background separation due to a quality that allows it to fit with reality, its performance is due to the effectiveness of patch-image dimensionality. The sparsity measure owing to the l₁-norm, on the other hand, shows less responsiveness in the case of a highly complex background with sharp edges than in the case of a smooth background. As a result, these strong edges may be sparse under l₁-norm reduction, implying that the strong edges may appear as targets under the constant global threshold of the IPI model, potentially increasing the false alarm rate. The existing IPI model has a low number of patches with strong edges, and thus, the strong edges could be viewed as an outlier. This is because nuclear NNM shrinks all SV by applying the same threshold and treats them equally. Furthermore, large SVs in a patch image convey more information than small SVs. Hence, we should punish the larger SVs less than the smaller SVs.

WNNM, on the other hand, gives different weights to SV according to importance to retain large SVs, which provide more edge information. In this work, the WNNM has been used in place of the NNM in the existing IPI model to constrain the complex background patch image (BPI) due to l1 norm minimization. The proposed model presented is addressed using the ADMM approach.

This study makes the following contributions:

➢

The infrared patch-image model via IPINCNWNNM–RPCA was proposed and was solved by the ADMM method.

➢

Extensive simulation was carried out that shows that the proposed scheme not only has good detection capabilities but also has good background estimation capabilities.

The structure of the remaining work is as follows: the materials and the procedures are presented in Section 2. In Section 3, the suggested IPNCWNNM–RPCA model is discussed in detail. The experimental evaluation and their interpretation are presented in Section 4, and the study closes in Section 5.

2. Materials and Methods

In the single-frame detection approach, several methods have been presented in the past: background perception, object saliency recognition, pattern classification, and IPI. A detailed summary of these critical methods is presented in Table 1 with their advantages and disadvantages.

2.1. Methods Based on Background Assumptions

This group of algorithms assumes that the background transition is slow and that the local region’s pixel correlation is fairly strong. These methods are very simple and easy to execute. The two-dimensional least (TDLMS) mean square [5], max–median, and max–mean [6] algorithms fall into this category, but these are not efficient in dense clouds. Improved approaches such as the edge-directed TDLMS filter [7] have been developed.

TDLMS based on neighborhood analysis [8] and an edge component-based bilateral filter [9] that predicts edge direction and maintains edges have shown better results. Top-hat filter and toggle-contrast filter [10,11], both based on morphological processes, suffer from the same issue discussed above.

2.2. Methods Based on Object Saliency Identification

These approaches use the difference of Gaussian presented by Wang et al. [12] to compute the saliency map. The weighted local coefficient of variation (WLCV) was presented by Rao et al. [13]. Chen et al. [3] presented a local contrast measure (LCM) that computes the saliency map. The improved version (ILCM) was proposed by Han et al. [14]. This category includes methods that consider the target more critical than the background. In addition, these methods are based on human visual phenomena. The work proposed in [15,16] uses the Laplacian of Gaussian (LOG) in target detection. The work proposed in [12] uses difference of Gaussian (DOG) for the saliency map calculation, and in another work, Han et al. [14] proposed the upgraded DOG filter to compute the saliency map. More work is available in the literature that fits this category [17,18].

2.3. Pattern Classification Based Methods

The approaches under this class are binary classification methods where the background and the targets are separated based on the patch information. These methods are inspired by the work of M. Turk and A. Pentland [19] and J. Write at al. [20] and founded on the principle of face recognition using PCA. The leading methods for projection are PCA [21], probabilistic PCA [22], nonlinear PCA [23], and sparse [24,25] representation. From the vast background, Wang et al. [25] constructed a method. The target and the background patch were separated using an adaptive weight in [26]. One distinct problem with these methods is the need for dictionary samples, which take a long time to process, and they also need a large dataset to perform.

2.4. Patch Image-Based Methods

The motivation for this class is the work in BM3D [27] and BM4D [28]. The method of this category creates dataset patches. The first study in this approach for target background separation was by Gao et al. [1]. By applying this technique, low-rank and sparse matrices were recovered by turning the IPI model into an optimization problem. This technique has the benefit of not requiring the usage of substantial dictionary samples.

Although the IPI model produces positive results, it has a serious problem with the l₁-norm sparsity minimization, which involves a trade-off between reducing the dim target and maintaining the strong edges of the image. Y. He et al. [29,30] conducted two other works based on the low-rank and sparse representations paradigm. Dai et al. [31] developed a new non-negative IPI model that estimates the background correctly and precisely while preserving. Similar work was proposed by Rawat et al. in [32], and Dai et al. [33,34] proposed an approach in which the prior structural knowledge is embedded in the background image. This approach leads to the complexity of calculation and has the rank computation issue. To tackle this, Gao et al. [35] suggested reweighted IPI (ReWIPI), which is based on the work in [36] to confine the BPI while keeping the background edge information. However, due to insufficient weight adjustment, this may give inaccurate SVD estimates.

In [37], a suggestion was made to add inherent smoothness to the BPI using TV regularization, and principal component pursuit (TV–PCP) and non-convex rank approximation minimization [38,39,40] are some recent breakthroughs in IPI-based techniques. Small target recognition is exceedingly difficult due to the object’s small size. Current IPI tactics, on the other hand, have performed nicely; however, some issues need to be addressed.

Table 1. Details of the key state-of-the-art target detection methods.

References	Publication Year	Method Name	Advantages	Disadvantages
Methods based on Background spatial consistency
M.M. Hadhoud and D.W. Thomas [5]	1998	TDLMS	This method is very simple to use for purposes like reducing noise and improving the object of interest.	In a noisy environment, it fails to perform.
S.D. Deshpande et al. [6]	1999	Max–median and max–mean	These methods are very simple to use for purposes like reducing noise and improving the object of interest.	In addition to the targets, these methods also enhance the strong cloud.
T.-W. Bae et al. [7]	2012	TDLMS edge-directional filter	Applies filtering to preserve the edges by estimating the direction.	In a noisy environment, it fails to perform.
Y. Cao et al. [8]	2008	Neighborhood-based analysis of TDLMS filter	The process computes the edge direction and preserves the edge based on neighbor information.	In a noisy environment, it fails to perform.
T.-W. Bae & K.-I. Sohng [9]	2010	Bilateral filter according to edge component	The process estimates the edge information based on bilateral filters.	In a noisy environment, it fails to perform.
R. Fortin and J. Rivest M. Zeng et al.,X. Bai et al. [10,11,41,42]	1996, 2006, 2012, 2010	Morphological-based methods, top-hat filter, and toggle contrast	These methods are very simple to use.	It is necessary to have a well-designed filter that can meet the desired qualities.
Methods based on Target saliency
Kim et al. [15] and Shao et al. [16]	2012	LOG	The primary purposes of these methods are to reduce noise and improve the object of interest.	Does not work well with very small or insignificant objects.
Wang et al. [12]	2012	Difference of Gaussian	Improves the target intensity and suppresses the clutter.	Does not work well with very small or insignificant objects.
Han et al. [14]	2016	Gabor filter	Improves the target intensity and suppresses the clutter.	Does not work well with very small or insignificant objects.
Chen et al. [3]	2014	LCM	Utilizes the local contrast information.	Does not work well with very small or insignificant objects.
Rao et al. [13]	2021	WLCV	Utilizes the weighted saliency map information.	Does not work well with very small or insignificant objects.
Yu et al. [17]	2022	Multiscale local contrast learning	Utilizes the local contrast information.	Does not work well with very small or insignificant objects.
Y. Wei et al. [18]	2016	MPCM	Utilizes the multi-patch information and the local contrast information.	Does not work well with very small or insignificant objects.
Small Target detection using patch-level
T. Hu et. al. [21], Y. Cao, [22], Liu et al. [23] C., Z.-Z. Li et al. and Wang et al. [24,25]	2010, 2008, 2005, 2012, 2014, 2015	PPCA, NLPCA, KPCA, (SR), and sea-sky background dictionary	Perform well when it comes to targeting background classification under noise.	The downsides of these systems include that each overlapped patch must be projected into a dictionary and that reconstructing the object of interest is a time-consuming process.
Small Target detection using patch-image level
Gao et al. [1], Rawat et al. [31,32,33,34] Gao et. [35,37,38,39]	2013, 2022, 2017, 2017, 2017, 2017, 2017, 2019,	IPI, TV-PSMSV, NIPPS, ReWIPI), RIPT, TV-PCP, NRAM, PSTN	In a complex clutter scene, these approaches display significant target- background suppression.	Compositionality is high in this case. Second, the l1 norm-based approach is used.

Table 1. Details of the key state-of-the-art target detection methods.

References	Publication Year	Method Name	Advantages	Disadvantages
Methods based on Background spatial consistency
M.M. Hadhoud and D.W. Thomas [5]	1998	TDLMS	This method is very simple to use for purposes like reducing noise and improving the object of interest.	In a noisy environment, it fails to perform.
S.D. Deshpande et al. [6]	1999	Max–median and max–mean	These methods are very simple to use for purposes like reducing noise and improving the object of interest.	In addition to the targets, these methods also enhance the strong cloud.
T.-W. Bae et al. [7]	2012	TDLMS edge-directional filter	Applies filtering to preserve the edges by estimating the direction.	In a noisy environment, it fails to perform.
Y. Cao et al. [8]	2008	Neighborhood-based analysis of TDLMS filter	The process computes the edge direction and preserves the edge based on neighbor information.	In a noisy environment, it fails to perform.
T.-W. Bae & K.-I. Sohng [9]	2010	Bilateral filter according to edge component	The process estimates the edge information based on bilateral filters.	In a noisy environment, it fails to perform.
R. Fortin and J. Rivest M. Zeng et al.,X. Bai et al. [10,11,41,42]	1996, 2006, 2012, 2010	Morphological-based methods, top-hat filter, and toggle contrast	These methods are very simple to use.	It is necessary to have a well-designed filter that can meet the desired qualities.
Methods based on Target saliency
Kim et al. [15] and Shao et al. [16]	2012	LOG	The primary purposes of these methods are to reduce noise and improve the object of interest.	Does not work well with very small or insignificant objects.
Wang et al. [12]	2012	Difference of Gaussian	Improves the target intensity and suppresses the clutter.	Does not work well with very small or insignificant objects.
Han et al. [14]	2016	Gabor filter	Improves the target intensity and suppresses the clutter.	Does not work well with very small or insignificant objects.
Chen et al. [3]	2014	LCM	Utilizes the local contrast information.	Does not work well with very small or insignificant objects.
Rao et al. [13]	2021	WLCV	Utilizes the weighted saliency map information.	Does not work well with very small or insignificant objects.
Yu et al. [17]	2022	Multiscale local contrast learning	Utilizes the local contrast information.	Does not work well with very small or insignificant objects.
Y. Wei et al. [18]	2016	MPCM	Utilizes the multi-patch information and the local contrast information.	Does not work well with very small or insignificant objects.
Small Target detection using patch-level
T. Hu et. al. [21], Y. Cao, [22], Liu et al. [23] C., Z.-Z. Li et al. and Wang et al. [24,25]	2010, 2008, 2005, 2012, 2014, 2015	PPCA, NLPCA, KPCA, (SR), and sea-sky background dictionary	Perform well when it comes to targeting background classification under noise.	The downsides of these systems include that each overlapped patch must be projected into a dictionary and that reconstructing the object of interest is a time-consuming process.
Small Target detection using patch-image level
Gao et al. [1], Rawat et al. [31,32,33,34] Gao et. [35,37,38,39]	2013, 2022, 2017, 2017, 2017, 2017, 2017, 2019,	IPI, TV-PSMSV, NIPPS, ReWIPI), RIPT, TV-PCP, NRAM, PSTN	In a complex clutter scene, these approaches display significant target- background suppression.	Compositionality is high in this case. Second, the l1 norm-based approach is used.

3. The Proposed Method

A single-frame image can be modeled in the following way:

$$f_D\left(a,b\right)= f_T\left(a, b\right)+f_B\left(a,b\right)+f_N\left(a,b\right)$$

(1)

where $f_D\left(a,b\right)$—real image, $f_B\left(a,b\right)$—background image, $f_T\left(a,b\right)$—target image, $f_N\left(a,b\right)$—noise image, and $\left(a,b\right)$—location of pixels coordinates in the taken image. The performance of RPCA in separating the background and the target image [43] motivated Gao et al. [1] to design an IPI model. The RPCA technique is used to reformulate target and background separation into an optimization problem by dividing the background into a low-rank matrix and the target into a sparse matrix. The sparse target image patch matrix T and the low-rank BPI matrix B are the two matrices that make up the patch-image matrix D in the IPI model. These matrices are then separated using RPCA as follows:

$$\underset{B,T}{\min }\Vert B{\Vert }_{\ast }+\lambda \Vert T{\Vert }_1, s.t D=B+T$$

(2)

where, $\Vert .{\Vert }_{\ast }$ is the matrix’s nuclear norm, which is calculated by adding SV, and $\Vert .{\Vert }_1$ is the l₁-norm, which is defined by $\Vert X{\Vert }_1={{\displaystyle \sum }}_{ij}\left|X_{ij}\right|$. A method known as accelerated proximal gradient [20] can be used to solve the convex optimization issue described in [1].

Although the IPI model has proven successful in target background separation due to a quality that allows it to fit with reality, its performance is due to the effectiveness of patch-image dimensionality. The sparsity measure owing to the l₁-norm, on the other hand, shows less responsiveness in the case of a highly complex background with sharp edges than in the case of a smooth background. As a result, these strong edges may be sparse under l₁-norm reduction, implying that the strong edges may appear as targets under the constant global threshold of the IPI model, potentially increasing the false alarm rate. We know that the IPI model has a low number of patches with strong edges, and thus, strong edges could be viewed as an outlier, which is a goal. This is because NNM shrinks all singleton values with the same threshold and treats them all equally. Furthermore, large single values in a patch image convey more information than tiny SVs; hence we should punish larger SVs less than small SVs. As previously stated in section two, WNNM is utilized in place of NNM, which assigns various weights to different SVs in order to retain the large SVs, which provide more edge information. As mentioned in Equation (1), the infrared image in the patch domain can be modeled as a linear combination of background and the target image patch as given below; for simplicity, we are neglecting the noise component:

$$D=B+T$$

(3)

3.1. Background Patch-Image

Figure 1a, where the SVs of all BPIs trend towards zero, illustrates the BPI’s strong association with both local and nonlocal patches, as was already indicated. The BPI is created using the nuclear norm. However, because the nuclear norm handles all SV identically, it is not always viable to anticipate the background patch picture using it. As a result, they shrink at the same rate, and high SVs, which contain more information, are penalized more severely than small SVs. As a result, instead of using the nuclear norm to assign weights to all the SV, we used the weighted SV threshold procedure to obtain the best background patch image.

Background matrix B’s weighted nuclear norm is defined as:

$$\Vert B{\Vert }_{w,\ast }={\displaystyle \sum }_i{w}_i{\sigma }_i\left(B\right)$$

(4)

where $w={\left[w_1\dots \dots \dots w_n\right]}^T$ is a weight that is allocated to i, and $w_i\ge 0$ is a nonnegative weight that is assigned to ${\sigma }_i\left(B\right)$_.

3.2. Small Target Patch-Image

The target size changes from 2 × 2 to 9 × 9, and its brightness is not fixed as it is with infrared photographs. Because the target is tiny in relation to the total image, we may think of the target patch image as a sparse matrix. The l₁ norm can be used as $\Vert \mathit{T}{\Vert }_1$ and here $\Vert \mathit{T}{\Vert }_1={{\displaystyle \sum }}_{ij}\left|{\mathit{T}}_{ij}\right|$.

3.3. Background Separation Solution for Small Target

The proposed IPI model employing weighted nuclear norm minimization via resilient principal component analysis can be offered as a result of reformulating Equation (2):

$$\underset{B,T}{\min }\Vert B{\Vert }_{w,\ast }+\lambda \Vert T{\Vert }_1, s.t D=B+T$$

(5)

where $\lambda$ is the parameter for weighting. ADMM is used to solve Equation (6), and its augmented Lagrange function is as follows:

$$\mathcal{L}\left(B,T,Y,\mu \right)=\Vert B{\Vert }_{w,\ast }+\lambda \Vert T{\Vert }_1+tr\left[Y^T\left(\mathrm{D}-\mathrm{B}-\mathrm{T}\right)\right]+\frac{\mu }{2}\Vert D-B-T{\Vert }^2{}_F$$

(6)

Here, Y is the Lagrange multiplier, $\mu$ is the scalar quantity, and tr$\left[.\right]$ is the trace operator. We can minimize $\mathcal{L}$ with respect to B and T using the inexact augmented Lagrange multiplier (IALM):

$$B^{k+1}=\arg \underset{B}{\mathit{min}\mathcal{L}(B^k},T^k,Y^k,{\mu }_k)$$

(7)

$$= \arg \underset{B}{min\Vert B{\Vert }_{w,\ast }+}\frac{\mu }{2}\Vert D+{\mu }_k{}^{-1}{Y}^k-T^{k+1}-B{\Vert }^2{}_F$$

$$T^{k+1}=\arg \underset{T}{\mathit{min}\mathcal{L}(B^{k+1}},T,Y^k,{\mu }_k)$$

(8)

$$\underset{T}{=\arg min \lambda \Vert T{\Vert }_1+}\frac{\mu }{2}\Vert D+{\mu }_k{}^{-1}{Y}^k-B^k-T{\Vert }^2{}_F$$

$$Y^{k+1}=Y^k+{\mu }_k\left(D-B^{k+1}-T^{k+1}\right)$$

(9)

The WNNM here can be addressed using the weighed nuclear norm proximal (WNNP) operator by changing it into a quadratic programming form with linear constraints:

$$\left\{\begin{array}{c}\widehat{X}=pro{x}_{\Vert .\Vert w,\ast }\left(\mathrm{Y}\right)\\ =\arg \underset{T}{\mathit{min}\Vert Y-X{\Vert }^2{}_F}+\Vert X{\Vert }_{w,\ast }\end{array}\right.$$

(10)

Theorem 1.

As Y$\in R^{mXn}$without loss of generality, assume that m$\gg n$, and let Y = UΣ$V^T$is the evaluated SVD of Y, here$\Sigma =\left(\begin{array}{c}diag\left({\sigma }_1,{\sigma }_2,{\sigma }_3\dots \dots {\sigma }_n\right)\\ 0\end{array}\right)\in R^{mXn}.$The global optimum of WNMP problem (11) can be expressed as$\widehat{X}=U\widehat{D}{V}^T$where D =$\left(\begin{array}{c}diag\left(d_1,d_2,d_3\dots \dots d_n\right)\\ 0\end{array}\right)$is a diagonal non-negetive matrix and$\left(d_1,d_2,d_3\dots \dots d_n\right)$is the ultimate answer to the convex optimization problem in Equation (11):

$$\begin{gathered} \begin{array}{c}min\\ d_1,d_2,d_3,\dots \dots d_n\end{array}{\displaystyle \sum }_{i=1}^n{\left({\sigma }_i-d_i\right)}^2+w_i{d}_{i,} \\ \mathrm{s}.\mathrm{t} d_1\ge d_2\ge \dots \dots d_n\ge 0 \end{gathered}$$

(11)

The closed-form optimum solution of WNNP can be obtained by weighted singular value threshold operation:

$$pro{x}_{\lambda \Vert .\Vert w,\ast }=D\left(Y\right)=U{S}_{\frac{w}{2}}(\mathrm{\Sigma })V^T$$

(12)

where Y = UΣ$V^T$ is the evaluated SVD of Y, $S_{\frac{w}{2}}$(Σ) is the soft threshold operator, and w is a weight vector.

$$S_{\frac{w}{2}}\left(\mathrm{\Sigma }\right)=\mathrm{S}\left(\mathrm{Y}\right)=\max \left({\mathrm{\Sigma }}_{ii }-\frac{w_i}{2}, 0\right)$$

(13)

$$B^{k+1}=D_{{\mu }_k{}^{-1}}\left(D-T^{k+1}+{\mu }_k{}^{-1}{Y}^k\right)$$

(14)

$$T^{k+1}=S_{{\mu }_k{}^{-1}}\left(D-B^k+{\mu }_k{}^{-1}{Y}^k\right)$$

(15)

Theorem 2.

The sequences$\left\{ X^k\right\} and\left\{ E^k\right\}$created by the algorithm below should satisfy the following conditions if the weights are arranged in increasing order:

(1)

$\underset{n\to \infty }{\lim }\Vert X^{k+1}-X^k{{\Vert }^2}_F=0$

(2)

$\underset{n\to \infty }{\lim }\Vert E^{k+1}-E^k{{\Vert }^2}_F=0$

(3)

$\underset{n\to \infty }{\lim }\Vert Y-E^{k+1}-X^{k+1}{{\Vert }^2}_F=0$

$$\begin{gathered} \mathrm{where} \underset{X,E}{\min }\Vert X{\Vert }_{\ast }+\lambda \Vert E{\Vert }_1, \\ s.t Y=X+E \end{gathered}$$

(16)

The symbol in the algorithm $\Vert .{\Vert }_2$ represents spectrum norm, $vec\left(.\right)$ represents the vector operator of the matrix, and $\Vert .{\Vert }_{inf}$ is the infinite norm of a vector.

In this work, we have utilized a reweighting approach as given below and utilized in the algorithm to improve the sparseness:

$${ \mathrm{w}}_{\mathrm{i}}{}^{\mathrm{l}}=\frac{\mathrm{C} }{{\mathsf{\sigma }}_{\mathrm{i}}\left({\mathrm{X}}_{\mathrm{l}}\right)+\theta }$$

(17)

Here, C represents a positive regularization parameter and $\theta$ represents a small positive to take care of the dividing by zero problem.

$$\begin{gathered} {\mathsf{\sigma }}_{\mathrm{i}}\left(X^{\ast }\right)=\left\{\begin{array}{c} 0 if c_2<0\\ \begin{array}{c}\frac{c_1+\sqrt{c_2}}{2}\end{array} if c_2\gg 0\end{array}\right. \\ \mathrm{Where} X^{\ast }=U\Sigma V^T \end{gathered}$$

(18)

3.4. Separation Model for Target-Background Model

Figure 2 above depicts the target-background division paradigm in its entirety. The entire procedure can be summarized as follows:

3.4.1. Creation of the Patch-Image Form Input

This is the initial phase, when an infrared patch image called D is created using the original image $f_D$ from the image sequence. A sliding window moves from left to right first and then moves down from top to bottom to create the patch images.

3.4.2. Target-Background Separation

In the second phase, the input patch image is processed using Algorithm 1 to fragment it into two matrices, first B and then T.

Algorithm 1 Solving IPNWNNM-RPCA via ADMM

Input: Real patch image D, weighting parameter $\lambda ,w$.

Output:$({\mathit{B}}^{\mathit{k}}, {\mathit{T}}^{\mathit{k}})$

Initialize:$B^0=T^0=0,Y^0=\frac{D}{\max \left(\Vert D{\Vert }_2, M\Vert vec\left(D\right){\Vert }_{inf}\right)},$

$\rho =1.05,\epsilon ={10}^{-7},k=0,\theta >0,{\mu }_0=\frac{1}{\Vert D{\Vert }_2},{\mu }_{max}={10}^7;$

While (not converged) do

1.  Correct the other and Change the B by

${\mathit{B}}^{\mathit{k}+\mathbf{1}}={\mathit{D}}_{{\mathbf{\mu }}_{\mathit{k}}{}^{-\mathbf{1}}}\left(\mathit{D}\mathbf{-}{\mathit{T}}^{\mathit{k}+\mathbf{1}}+{\mathbf{\mu }}_{\mathit{k}}{}^{-\mathbf{1}}{\mathit{Y}}^{\mathit{k}}\right) ;$

2.  Correct the other and Change the T by

${\mathit{T}}^{\mathit{k}+\mathbf{1}}={\mathit{S}}_{{\mathbf{\mu }}_{\mathit{k}}{}^{-\mathbf{1}}}\left(\mathit{D}-{\mathit{B}}^{\mathit{k}}+{\mathbf{\mu }}_{\mathit{k}}{}^{-\mathbf{1}}{\mathit{Y}}^{\mathit{k}}\right)$;

3.  Correct the other and Change the Y by

${\mathit{Y}}^{\mathit{k}}+{\mathbf{\mu }}_{\mathit{k}}\left(\mathit{D}-{\mathit{B}}^{\mathit{k}+\mathbf{1}}-{\mathit{T}}^{\mathit{k}+\mathbf{1}}\right)$;

4.  Update by µ

${\mu }_{k+1}=\rho \ast {\mu }_k$;

5.  Check convergence condition

$\frac{\Vert \mathit{D}-{\mathit{B}}^{\mathit{k}+\mathbf{1}}-{\mathit{T}}^{\mathit{k}+\mathbf{1}}{{\Vert }^{\mathbf{2}}}_{\mathit{F}}}{\Vert \mathit{D}{\Vert }_{\mathit{F}}}<\mathit{\epsilon }$

6.  Update k

k++;

end

3.4.3. Regeneration of the Target and Background Images

In the third phase, the proposed method reconstructs $f_T$, and $f_B$ from the target patch images and the background. The whole process can be accomplished using the technique outlined in [1].

3.4.4. Segmentation Process

Now the final touch is initiated where some final processing to enhance the quality of target image is performed to run the adaptive thresholding scheme as described in [1].

The adaptive threshold evaluation is performed using Equation (19):

$$t_{up}=\max \left(v_{min},\overline{f_T}+k\sigma \right)$$

(19)

where $\overline{f_T}, \sigma$ represents the average and the standard deviation of $f_T$, and k, respectively, and $v_{min}$ denotes the constant, which is taken as an empirical value. If $f_T\left(x,y\right)>t_{up}$, then pixels are part of the target image; otherwise, they are the part of the background image.

4. Experimental Result Analysis

This section presents a detailed experimental evaluation of the IPNCWNNM–RPCA model and finally compares its performance with that of the referenced state-of-art methods.

4.1. Parameter Settings, Baseline Methods, and Evaluation Indicators Metrics

We analyzed real infrared single images and sequences with a range of backgrounds, including water, sky, cloud, and land, using the proposed IPNCWNNM–RPCA model as well as six additional cutting-edge techniques. Additionally, the approach described in [1] is used to make the synthetic images.

Figure 3 shows representative images from the image sequences, whereas

Table 2. Summary of the real image sequence data.

Sequences	Target Type	Image Size	No of Frames	Background Image Features	Target Image Features
1	Small ship	256 × 200	30	Dense sea-sky	Small size (SS) Varying size Large imaging distance (LID)
2	An airplane	256 × 200	250	High dense clouds with less local contrast	Small size Varying size LID
3	Two target	256 × 200	250	Changing background	LID SS Low SCR
4	Copter	128 × 128	100	Changing background	LID Low SCR SS
5	Ship	128 × 128	200	Changing background	SS A one-to-two target
6	An airplane	280 × 228	250	High dense clouds with less local contrast	LID SS

provides a detailed description of these images.

Table 3. Summary of the different parameters used in the evaluations.

Sr. No.	Techniques	Parameters
1	Max–median [6]	Filter = 5 × 5
2	Max–mean [6]	Filter = 5 × 5
3	IPI [1]	Sliding step = 10, tolerance error $\epsilon ={10}^{-7}$, Patch size = 50 × 50$, \lambda =\frac{1}{\sqrt{m}}$
4	NIPPS [31]	Sliding step = 10, Patch size = 50 × 50, r = ${10}^{-3},$$\mathrm{L}=2, \epsilon ={10}^{-7}$$, \lambda =\frac{\mathrm{L}}{\sqrt{\min \left(\mathrm{m},\mathrm{n}\right)}}$$, \rho$ = 1.5
5	Top-Hat [11]	Filter shape = square, square size = 3 × 3
6	RPCA [26]	sliding step = 10, Patch size = 50 × $50, \epsilon ={10}^{-7}$, $\lambda =\frac{1}{\sqrt{m }}$
7	RIPT [35]	$\lambda =\frac{\mathrm{L}}{\sqrt{\min \left(\mathrm{m},\mathrm{n}\right)}}$, Patch size = 50 × 50, L = 1, h = 1, $\epsilon ={10}^{-7},$ sliding step = 10
8	PSTN [39]	$\lambda =\frac{\mathrm{L}}{\sqrt{\left(\max \left(\mathrm{n}1,\mathrm{n}2\right)\ast \mathrm{n}3\right)}}$, Patch size = 40 × 40, L = 0.6,$\rho$ = 1.05, sliding step = 40,$\epsilon ={10}^{-7}$
9	Infra small target detection based on nonconvex LP norm minimization (IPCWLP–RPCA [41]	$\mathrm{tolerance} \mathrm{error} \epsilon ={10}^{-7},\mathrm{C} =\sqrt{\mathrm{m}}, \theta =0.005,\rho$ = 1.05, $\mathrm{Patch} \mathrm{size} \mathrm{is} 50 \times 50, \lambda =\frac{1}{\sqrt{m}}$, sliding step is 10
10	Our Method	$\lambda =\frac{1}{\sqrt{m}}$$, \mathrm{Patch} \mathrm{size}=50 \times 50, \mathrm{C} =\sqrt{\mathrm{m}}$$, \mathrm{sliding} \mathrm{step}=10, \epsilon ={10}^{-7}, \theta =0.005, \rho$ = 1.05

gives the detailed parameters for several baseline approaches. The ADMM is used to solve all of the infrared patch-based approaches. All of the algorithms were developed in MATLAB 2015-a on a computer with a 2.4 GHz processor and 8-GB of RAM.

4.2. Evaluation Indicators

This section describes how the background of an image can be suppressed as well as the target in an image can be improved. The suggested method’s performance is validated using two standards, namely, SCRG and BSF. These indicators are described in depth in [44] and can be stated as follows:

$$\mathrm{BSF}=\frac{C_{in}}{C_{out}}, \mathrm{SCRG}=\frac{{\left(\raisebox{1ex}{$\mathrm{S}$}\!\left/ \!\raisebox{-1ex}{$\mathrm{C}$}\right.\right)}_{\mathrm{out}}}{{\left(\raisebox{1ex}{$\mathrm{S}$}\!\left/ \!\raisebox{-1ex}{$\mathrm{C}$}\right.\right)}_{\mathrm{in}}}$$

(20)

where S and C denote the signal amplitude and clutter standard deviation (SD) and _in and _out in the formula denote the input original image and output target image, respectively. Before and after the image is analyzed, SCRG reports the signal’s amplification result. When no information about the target is available, BSF assigns a level of suppression. As a result, it is expected that both indicators will have a high value in order to improve efficiency. The response of various methods can be validated using a metric known as ROC. The changing connection between the likelihood of target detection Pd and the false alarm rate Pf is shown by this curve [45], and it may be put this way:

$$\left\{\begin{array}{c}{\mathrm{P}}_{\mathrm{f}}=\frac{\mathrm{Number} \mathrm{of} \mathrm{false} \mathrm{alarms}}{\mathrm{Total} \mathrm{count} \mathrm{of} \mathrm{pixels} \mathrm{in} \mathrm{the} \mathrm{complete} \mathrm{image}}\\ {\mathrm{P}}_{\mathrm{d}}=\frac{\mathrm{Count} \mathrm{of} \mathrm{detected} \mathrm{pixels} }{\mathrm{Count} \mathrm{of} \mathrm{original} \mathrm{target} \mathrm{pixels}}\end{array}\right.$$

(21)

As illustrated in Figure 4, all of the aforementioned indicators are computed in a small local region. If a tiny target’s size is a × b, the background rectangle’s size is (a + 2d) × (b + 2d), where d is a constant equal to 20 pixels.

4.3. Results of Experiments on Single Infrared Images

In the outcome analysis, we used a dataset containing more than 1500 single infrared images with different backgrounds. Each image has a maximum of two targets, and we used them to demonstrate the background suppression capabilities of different approaches in different background environments. Figure 5a–e show representative images and findings from different perspectives. When the background image is smooth, as in the images shown in Figure 5b–d, the max–mean, max–median, and top-hat approaches can identify the target. However, they fail to enhance the target when the background is complex, as seen in Figure 5e. In addition, the top-hat approach relies on good filter selection to properly detect the target; otherwise, it will fail. Similarly, baseline approaches such as IPI as shown in Figure 5e identify the target pretty effectively. However, when there is much clutter in the background, the methods fall short because they overshrink the image, causing the non-target element to be recognized instead of the genuine target.

The RPCA method as shown in Figure 6a is capable of detecting the target. However, its performance is hampered by the global threshold-holding parameter. NIPPS shown in Figure 6b outperforms other baseline approaces in terms of compassion. However, it suffers in a complicated background because the rank of the patch image matrix must be predicted accurately. Although RIPT and NRAM, shown in Figure 6c,d, performed very well, these methods fail to perform well in the presence of heavy noise. Moreover, these methods suffer from a matrix rank issue.

The suggested IPNCWNNM–RPCA technique as presented in Figure 6e smooths the clutter background by employing weighted nuclear norm singular value minimization, in which each singular value is given a variable weight, and highly informative SVs are shrunk less while less informative single values are shrunk more.

As a result of our method, strong edges such as corners and buildings may be predicted with ease, and the true candidate target can be identified quickly. The receiver operating curves (ROC) for the dataset’s single infrared image are presented in Figure 7a–f, indicating that our method has a better response when compared with other referenced methods.

In comparison with the other methods, the proposed IPNCWNNM–RPCA has impressive background suppression and target identification capabilities. The highest BSF and SCRG are in bold in

Table 4. BSF and the SCRG values obtained using the various approaches for each of the test images in Figure 4a. The best outcomes are indicated in bold.

Detection Methods	Evaluation Indicators	Image6	Image5	Image4	Image3	Image2	Image1
Max–Median	BSF	1.914	1.861	3.816	1.934	3.404	1.354
	SCRG	1.863	3.117	51.109	16.001	6.358	5.881
Top Hat	BSF	0.528	0.923	2.354	0.882	1.573	1.104
	SCRG	2.938	3.081	53.302	9.440	4.024	6.546
Max–Mean	BSF	1.521	1.167	3.249	1.714	1.795	1.185
	SCRG	6.640	2.117	36.456	10.211	2.532	4.460
RPCA	BSF	0.952	0.494	6.468	7.443	0.681	0.489
	SCRG	2.274	0.683	76.236	73.628	0.279	3.166
IPI	BSF	22.280	29.862	13.778	8.799	13.565	3.219
	SCRG	115.118	125.505	263.310	113.135	34.854	0.013
NIPPS	BSF	36.604	7.413	6.726	3.898	39.983	3.955
	SCRG	182.053	30.018	168.042	55.151	80.137	15.629
RIPT	BSF	13.638	26.180	10.155	10.340	5.210	4.734
	SCRG	71.826	107.088	196.948	87.306	16.036	18.458
IPCWLP—RPCA	BSF	1.60	2.55	1.37	3.23	1.40	1.29
	SCRG	9.31	96.89	115.04	74.74	7.69	188.50
IPNCWNNM–RPCA	BSF	40.290	101.232	3.226	15.132	2.551	1.600
	SCRG	194.540	14.860	74.835	125.127	106.124	21.213

, and the second highest values are in blue. Because time is such an essential component for any technique, IPNCWNNM–RPCA is slower than the other IPI-based methods. This is because a significant amount of time is spent creating patch images and reshaping them. The computing times for each approach are indicated in

Table 5. Comparative analysis of the costs and time involved in the computations.

Detection Methods	Top Hat	Max–Mean	Max–Median	I PI	RPCA	NIPPS	RIPT	NRAM	PSTN	IPCWLP-RPCA	IPNCWNNM–RPCA
Complexity	$\mathcal{O}$ (k2log k2M × N)	$\mathcal{O}$ (k2M × N)	$\mathcal{O}$ (k2M × N)	$\mathcal{O}$ (m × n2)	$\mathcal{O}$ (m × n2)	$\mathcal{O}$ (m × n2)	$\mathcal{O}$ (m × n2)	$\mathcal{O}$ (m × n2)	$\mathcal{O}({\mathrm{d}}_1{ \mathrm{d}}_2{\mathrm{d}}_{3 }$ $({\mathrm{d}}_{1 }{\mathrm{d}}_2$ $+{\mathrm{d}}_{2 }{\mathrm{d}}_2+{\mathrm{d}}_{1 }{\mathrm{d}}_2)$)	$\mathcal{O}$ (k × m × n2)	$\mathcal{O}$ (m × n2)
Time (s)	0.968	7.70	6.84	12.64	10.86	5.15	1.95	3.89	0.35	11.78	10.52

. Nonetheless, we will work to reduce this time constraint in the future.

4.4. Computational Complexity

The computational cost of running sequence No. 2 in Figure 5a is given in Table 5. The computing costs for the top-hat approach with a structuring size of k² and an image size of M × N is (k²log k²MN), and the max–mean and median are provided as M × N × k², respectively. We can observe that the SVD operations in the algorithm require a significant amount of time in all of the IPI model-based techniques. The cost of IPCWLP–RPCA is (k × m × n²), and for RPCA, IPI, NIPPS, and our technique, it is therefore given by $\mathcal{O}$ (m × n²) for the image patch size of m × n.

4.5. Infrared Image Sequences Yielded Experimental

The proposed method has been tested on various real infrared image sequences in a variety of environments. The targets in these image sequences are small and monotonous and have poor contrast. Furthermore, the images have sharp edges, which makes detection challenging. Eight state-of-the-art techniques, including max–mean [6], max–median [6], top-hat [11], IPI [1], NIPPS [31], RPCA [26], RIPT [35], and NRAM [38], have been compared with the proposed method in order to validate its efficacy. The Figure 8a displays the initial infrared image sequences, while Figure 8b–e and Figure 9a–e exhibit the results of the various baseline techniques.

Max–mean and max–median can find the small target when there is a smooth background but not when there is a complex background. Similarly, the top-hat technique is effective against a clean background but fails against one that is complex. Second, because the top-hat filter’s mask is determined by the target’s size, it is difficult hard to create a mask that matches the target’s size. IPI-based approaches are very effective at detecting tiny targets. However, they fail to perform better in cluttered backgrounds because they are unable to distinguish between real targets and strong edges, which they may incorrectly recognize as a target due to l₁-minimization.

The global weighting parameter is used in the RPCA method, which makes it difficult to detect small targets in a cluttered background. Furthermore, the NIPPS approach is reliant on knowing the rank of the matrix ahead of time, which is not always easy to predict. The suggested technique used weighted nuclear norm minimization, which gives SVs various weights and penalizes less informative SVs more than more informative SVs. Second, there are no restrictions on rank prediction ahead of time.

As a result, compared with existing baseline approaches, the suggested method not only suppresses the strong clutter background but it detects the small target better. The different SCRG values for the image sequences from the six state-of-the-art approaches and the suggested method are shown in

Table 6. SCRG values for the test images shown in Figure 8a.

Detection Methods	Evaluation Indicators	Image6	Image5	Image4	Image3	Image2	Image1
Max–Median	BSF	1.255	1.861	3.816	0.863	3.387	1.383
	SCRG	17.867	3.117	51.109	6.461	1.580	1.936
Top Hat	BSF	0.923	0.923	2.354	0.512	2.339	0.488
	SCRG	24.651	3.081	53.302	7.376	5.733	1.412
Max–Mean	BSF	1.195	1.167	3.249	0.747	3.895	1.295
	SCRG	17.393	2.117	36.456	5.415	1.708	1.765
RPCA		3.790	0.494	6.468	3.073	25.882	3.701
	SCRG	90.559	0.683	76.236	36.166	60.950	12.672
IPI	BSF	10.410	29.862	13.778	7.680	52.274	8.698
	SCRG	195.948	125.505	263.310	79.869	112.307	17.799
NIPPS	BSF	7.576	7.413	6.726	2.687	6.169	4.453
	SCRG	4.700	30.018	168.042	23.787	6.298	0.621
RIPT	BSF	14.874	0.896	3.125	3.101	7.124	3.440
	SCRG	0.038	0.062	24.799	9.308	4.835	0.476
NRAM	BSF	3.026	1.477	3.002	1.776	4.948	1.401
	SCRG	14.284	0.033	27.870	6.726	2.694	0.404
IPNCWNNM–RPCA	BS	18.138	41.475	14.482	8.678	4.022	5.564
	SCRG	27.860	0.394	134.26	95.494	89.265	18.298

.

The best SCRG shows that the target can be improved and is easily visualized. The largest SCRG and BSF are bold, while the second highest values are in blue. SCRG and BSF should be high enough for improved detection.

For the different real image sequences, Figure 10 shows the ROCs of the six approaches and the proposed method. Figure 10a shows that the proposed method has performed effectively and has achieved probability 1 for image sequences 1. The NIPPS approach has a low detection rate and has the lowest reaction in Figure 10b, whereas IPI comes in second. Figure 10c shows that the suggested method is late when compared with the IPI method, and again NIPPS has a low detection rate. Finally, we can see in Figure 10d,e the comparison of the proposed approach with the other methods. Moreover, from Figure 10f, it can be seen that our approach has performed nicely, although the IPI approach produces a strong response. In sum, the suggested method has responded nicely in easing the detection of the target object in the image.

Compared with the existing baseline approaches, the suggested method not only suppresses the strong clutter background but detects the small target better. The different SCRG values for the image sequences from the six state-of-the-art approaches and the suggested method are shown in Table 6.

4.6. Simulation Results for the Infrared Image Sequences with Noise

In the presence of noise, the proposed approach was tested on image sequences. Figure 11a depicts the original image sequences, while Figure 11b,c depict images with the noise of 10 and 20 standard deviations, respectively. Figure 11d,e show that in the presence of noise, the suggested technique suppresses the background and correctly detects the small target.

4.7. Simulation Results When Infrared Image Sequences Are Synthetic

On test datasets of infrared image sequences, we tested the suggested method’s performance. Actual infrared BPIs were used to construct a total of 100 synthetic test image sequences, each with 50 different-sized targets positioned at random locations throughout the background and varying clutter backgrounds. The whole preparation method for synthetic datasets is provided in [1]. In the assessment procedure, one and four target image sequences were employed. The original BPI sequences are shown in Figure 12a. Figure 12b depicts a single target in the original background, while Figure 12c depicts the outcome of the suggested method on a synthetic image. Figure 12d depicts the four targets in the real background, while Figure 12e depicts the outcome of the suggested method on a synthetic image. In the case of synthetic image sequences, we may say that the suggested technique is robust enough to detect the small targets.

4.8. Parameter Analysis

Three factors that are crucial for the robustness of the recommended technique under various background conditions have been used in this section: step size, patch size, and regulatory parameters. It is essential to make these adjustments in order to achieve the greatest outcomes. It is also crucial to remember that the criteria do not always produce the best overall outcome. ROC curves made for various picture sequences may be used to assess the effectiveness of the proposed approach utilising these parameters, as illustrated in Figure 13 in the section below.

4.8.1. Patch Size

We used varied patch sizes in the experiment to see how our approach performed, and we discovered that raising the patch size improves the target’s sparsity while simultaneously increasing the computation cost. During the experimental observation, we utilised patch sizes of 20, 30, 40, 50, and 60, taking into account the appropriate patch size for an enhanced response of the proposed model. The ROC curves for the two image sequences were produced, as seen in Figure 13a. The ROC curve demonstrates that increasing the image patch size significantly affects both detection effectiveness and computational cost.

Due to the loss of nonlocal information, patch size 60 with the proposed technique has inferior detection performance, which will surely make it more difficult to distinguish between the target and background. This suggests that for optimum performance, a patch size of 40 × 40 is ideal.

4.8.2. Step Size

The step size has to be tuned in the same manner as the patch size. During the experimental observation, the patch size was set to 40 × 40, and the step size was changed in increments of two units, yielding step sizes of 10, 12, 14, 16, and 18, respectively. The ROC curve on step size for the two picture sequences shown in Figure 13b indicates that using a small step can increase computation time and also negatively affects the detection performance of the suggested approach. Additionally, increasing the step size helps speed up processing. Our findings have led us to the conclusion that 12 is the ideal step size.

4.8.3. Controlling Parameter $\mathsf{\lambda }$

For the proposed method, $\lambda =\frac{\mathrm{L}}{\sqrt{\left(\max \left(\mathrm{n}1,\mathrm{n}2\right)\ast \mathrm{n}3\right)}}$, and lambda, L, is an important regulating parameter that controls both the background patch picture and the target patch image. A large L can cause an issue with overshrinking, but a small value will maintain the residue in the background picture and may even provide a misleading alert. In this experiment, we employed four distinct L values: L = 0.6, L = 0.8, L = 1.0, and L = 1.2. Compared with the other values, L = 0.8 generated superior results, as evidenced by the created ROC curves for two picture sequences utilizing these values shown in Figure 13c.

5. Conclusions

Due to the l₁ norm problem, the IPI model has difficulty constraining the background using the basic nuclear norm minimization. Because of this flaw, the non-target edges in the backdrop are mistaken for target spots. To overcome this problem and properly constrain the background patch image, the existing IPI model has been utilized in this work, which employs WNNM in conjunction with RPCA. In this model, the weights are applied to each singular value, and larger singular values are penalized less than smaller ones.

As a result, the provided model improves target recognition and background suppression. In comparison with the state-of-the-art approaches, the experimental findings show that the current IPNCWNNM–RPCA not only suppresses the clutter background but efficiently detects the object. The proposed model could further be improved in future by using the tensor norm instead of the nuclear norm or the WNN.