A Regularized Tseng Method for Solving Various Variational Inclusion Problems and Its Application to a Statistical Learning Model

Abstract

We study three classes of variational inclusion problems in the framework of a real Hilbert space and propose a simple modification of Tseng’s forward-backward-forward splitting method for solving such problems. Our algorithm is obtained via a certain regularization procedure and uses self-adaptive step sizes. We show that the approximating sequences generated by our algorithm converge strongly to a solution of the problems under suitable assumptions on the regularization parameters. Furthermore, we apply our results to an elastic net penalty problem in statistical learning theory and to split feasibility problems. Moreover, we illustrate the usefulness and effectiveness of our algorithm by using numerical examples in comparison with some existing relevant algorithms that can be found in the literature.

1. Introduction

Variational inclusion problems have widely been studied by researchers because of their many valuable applications and generalizations. It is well known that many problems in applied sciences and engineering, mathematical optimization, machine learning, statistical learning, and optimal control can be modeled as variational inclusion problems. See, for example, [1,2,3] and references therein. In addition, under some assumptions, such problems involve many important concepts in applied mathematics, such as convex minimization, split feasibility, fixed points, saddle points, and variational inequalities; see, for example, [4,5,6,7].

Let $\mathcal{H}$ be a real Hilbert space, and let $S:\mathcal{H}\rightrightarrows \mathcal{H}$ and $T:\mathcal{H}\to \mathcal{H}$ be maximal monotone and monotone operators, respectively. The variational inclusion problem (VIP) is to

$$\mathrm{find}u\in \mathcal{H}\mathrm{such}\mathrm{that}0\in (T+S)u.$$

(1)

A foremost method proposed for solving (1) is the forward-backward splitting method (FBSM) [2]. The FBSM operates as follows:

$$x_{n+1}={(I+\lambda S)}^{-1}(x_n-\lambda T{x}_n)\forall n\in \mathbb{N},$$

where $\lambda \in (0,\frac{2}{L})$ and $I:\mathcal{H}\to \mathcal{H}$ is the identity operator. The FBSM was proved to generate sequences that converge weakly to a solution of (1) under the assumption that T is $\frac{1}{L}$-cocoercive (or inverse strongly monotone). Aside from the cocoercivity assumption, convergence is only guaranteed under a similar strong assumption such as the strong monotonicity of $S+T$ [8]. Interested readers could consult [9] for results on finding zeros of sums of maximal monotone operators using a similar forward-backward scheme.

As an improvement on the FBSM, Tseng [10] proposed the modified forward-backward splitting method (MFBSM) (also called the forward-backward-forward splitting method) for solving the VIP for a more general case, where T is monotone and L-Lipschitz continuous. The MFBSM has the following structure:

$$\left\{\begin{array}{c}{y}_n={(I+\lambda S)}^{-1}(x_n-\lambda T{x}_n)\hfill \\ x_{n+1}=y_n-\lambda (T{y}_n-T{x}_n),n\in \mathbb{N}.\hfill \end{array}\right.$$

A weak convergence theorem was proved for this algorithm. While the implementation of the FBSM requires the prior knowledge of the Lipschitz constant of T, which makes the algorithm a bit restrictive, the MFBSM uses line search techniques to circumvent the onerous task of estimating the Lipschitz constant. Be that as it may, line search techniques are computationally expensive, as they require several extra computations per iteration; see, for example, [11]. For the inexact and the stochastic versions of the MFBSM, respectively, please see [12,13].

Motivated by the need to reduce the computational burden associated with the line search technique, and by the fact that strong convergence is more desirable than weak convergence in infinite dimensional Hilbert spaces and in applications, some authors have incorporated existing hybrid techniques in the MFBSM and have proposed hybrid-like strongly convergent methods with self-adaptive step sizes. See, for instance, [11,14] and references therein.

Let $S:\mathcal{H}\rightrightarrows \mathcal{H}$ and $T_i:\mathcal{H}\to \mathcal{H}$ be maximal monotone and monotone operators, respectively, where $i\in \left[I\right]:=\{1,2,\cdots ,I\}$. We recall the modified variational inclusion problem (MVIP) introduced in [15]:

$$\mathrm{find}u\in \mathcal{H}\mathrm{such}\mathrm{that}0\in \sum _{i\in \left[I\right]}(a_i{T}_i+S)u,$$

(2)

where $a_i\in (0,1)$ and ${\displaystyle \sum _{i\in \left[I\right]}}{a}_i=1$. Obviously, the MVIP is more general than the VIP in the sense that if $T_i=T\forall i\in \left[I\right]$, then the MVIP becomes the VIP. It is easy to see that if $x\in {\cap }_{i\in \left[I\right]}{(T_i+S)}^{-1}\left(0\right)$, then $0\in {\displaystyle \sum _{i\in \left[I\right]}}(a_i{T}_i+S)x$. However, the converse is not true in general.

In addition, the MVIP is more general than the following common variational inclusion problem, which has recently been studied in [16]:

$$\mathrm{find}0\in {\cap }_{i\in \left[I\right]}(T_i+S)u.$$

(3)

In view of its generality, the MVIP has recently attracted the attention of some authors who studied it and proposed algorithms for solving it. The authors of [17] studied the problem for the case where the $T_i$s are inverse strongly monotone. They proposed a Halpern-inertial forward-backward splitting algorithm for solving the problem and proved a strong convergence theorem. Moreover, the authors of [18] studied the MVIP in the case where the $T_i$s are monotone and Lipschitz continuous and designed a modified Tseng method for solving it. By using some symmetry properties, they proved the weak convergence of the method they proposed. Furthermore, they provided a formulation of an image deblurring problem as an MVIP and used their results in order to solve this problem.

On the other hand, motivated by the work [16], the authors of [19] have recently studied the following common variational inclusion problem:

$$\mathrm{find}0\in {\cap }_{i\in \left[I\right]}(T_i+S_i)u,$$

(4)

where $S_i:\mathcal{H}\rightrightarrows \mathcal{H}$ are maximal monotone operators and $T_i:\mathcal{H}\to \mathcal{H}$ are monotone Lipschitz continuous operators. They combined the inertial technique, Tseng’s method, and the shrinking projection method to design an iterative algorithm that solves (4).

Motivated by the above studies, in this paper, we propose a unified simple modification of the MFBSM, which we call the regularized MFBSM (RMFBSM), for solving (2)–(4) and establish a strong convergence theorem for the sequences it generates. To realize our objectives, we first examine a regularized MVIP and study the solution nets it generates. The novelty of our scheme lies in the fact that, unlike the existing modifications of the MFBSM in the literature, it yields strong convergence while preserving the two-step structure of the MFBSM. In the case where $T_i=T$ and $S_i=S$ for all $i=1,2,\cdots ,I$, we apply our result to a statistical learning model and to split feasibility problems.

The organization of our paper is as follows: In Section 2, we recall some useful definitions and preliminary results, which are needed in our study and in the convergence analysis of our algorithm. In Section 3, we propose the regularized MFBSM and establish a strong convergence theorem for it. In Section 4, we give some applications of our main results. In Section 5, we present numerical examples to illustrate our method and compare it with some existing related algorithms in the literature. We conclude with Section 6.

2. Preliminaries

We start this section by stating some notations and recalling a number of important definitions.

Let C be a nonempty, closed, and convex subset of a real Hilbert space $\mathcal{H}$, the inner product and induced norm of which are denoted by $\langle ·,·\rangle$ and $\parallel ·\parallel$, respectively. We denote by ‘$u_n\rightharpoonup u$’ and ‘$u_n\to u$’ the weak and the strong convergence, respectively, of the sequence $\left\{u_n\right\}$ to a point u. The following identity is well known:

$${\parallel u\pm v\parallel }^2={\parallel u\parallel }^2\pm 2\langle u,v\rangle +{\parallel v\parallel }^2\forall u,v\in \mathcal{H}.$$

Definition 1

([1,20]). A mapping $T:\mathcal{H}\to \mathcal{H}$ is said to be

 (i)

L-Lipschitz continuous if there exists a constant $L>0$ such that

$$\parallel Tu-Tv\parallel \le L\parallel u-v\parallel \forall u,v\in \mathcal{H};$$

 (ii)

β-strongly monotone if there exists a constant $\beta >0$ such that

$$\langle Tu-Tv,u-v\rangle \ge {\beta \parallel u-v\parallel }^2\forall u,v\in \mathcal{H};$$

 (iii)

β-inverse strongly monotone if there exists a constant $\beta >0$ such that

$$\langle Tu-Tv,u-v\rangle \ge {\beta \parallel Tu-Tv\parallel }^2\forall u,v\in \mathcal{H};$$

 (iv)

monotone if

$$\langle Tu-Tv,u-v\rangle \ge 0\forall u,v\in \mathcal{H};$$

 (v)

hemicontinuous if for every $u,v,w\in \mathcal{H}$, we have

$$\underset{\alpha \to 0}{lim}\langle T(u+\alpha v),w\rangle =\langle Tu,w\rangle .$$

Remark 1.

In view of the above definitions, it is clear that every strongly monotone mapping is monotone. In addition, Lipschitz continuous mappings are hemicontinuous.

Let $S:\mathcal{H}\rightrightarrows \mathcal{H}$ be a set-valued operator. The graph of S, denoted by $gr\left(S\right)$, is defined by

$$gr\left(S\right):=\left\{\right(u,v)\in \mathcal{H}\times \mathcal{H}:v\in Su\}.$$

The set-valued operator S is called a monotone operator if ∀$(u,v),(y,z)\in gr\left(S\right)$, $\langle u-y,v-z\rangle \ge 0$ and a maximal monotone operator if the graph of S is not a proper subset of the graph of any other monotone operator. For a maximal monotone operator S and $\lambda >0$, the resolvent of S is defined by

$$J_{\lambda S}:={(I+\lambda S)}^{-1}:\mathcal{H}\to \mathcal{H}.$$

It is well known that $J_{\lambda S}$ is firmly nonexpansive (in particualr, nonexpansive).

For each $u\in \mathcal{H}$, there exists a unique nearest element, denoted by $P_{C}u\in C$ and called the metric projection of $\mathcal{H}$ onto C at u. That is,

$$\parallel u-P_{C}u\parallel \le \parallel u-v\parallel \forall v\in C.$$

The indicator function of C, denoted by $i_C$, is defined by

$$i_C\left(u\right):=\left\{\begin{array}{cc}0,\hfill & \mathrm{if}u\in C,\hfill \\ +\infty ,\hfill & \mathrm{if}u\notin C.\hfill \end{array}\right.$$

Recall that the subdifferential $\partial f$ of a proper convex function f at $u\in \mathcal{H}$ is defined by

$$\partial f\left(u\right):=\{z\in \mathcal{H}:f(u)+\langle z,y-u\rangle \le f(y),\forall y\in \mathcal{H}\}.$$

The normal cone of C at the point $u\in \mathcal{H}$, denoted by $N_C\left(u\right)$, is defined by

$$N_C\left(u\right):=\left\{\begin{array}{cc}\{x\in \mathcal{H}:\langle x,y-u\rangle \le 0\forall y\in C\},\hfill & \mathrm{if}u\in C,\hfill \\ \varnothing ,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

We know that $\partial i_K$ is a maximal monotone operator, and we have $\partial i_K=N_K$. Furthermore, for each $\lambda >0$,

$$P_{C}u={(I+\lambda \partial i_K)}^{-1}u.$$

(5)

The following important lemmata are useful in our convergence analysis.

Lemma 1

([21]). Let $\mathcal{H}$ be a real Hilbert space. Let $S:\mathcal{H}\rightrightarrows \mathcal{H}$ be a maximal monotone operator and let $T:\mathcal{H}\to \mathcal{H}$ be a monotone and Lipschitz continuous mapping. Then, the mapping $M=S+T$ is a maximal monotone mapping.

Lemma 2

([22]). Let $\mathcal{H}$ be a real Hilbert space. Let K be a nonempty, closed, and convex subset of $\mathcal{H}$, and let $\mathcal{F}:\mathcal{H}\to \mathcal{H}$ be a hemicontinuous and monotone operator. Then, $\overline{u}$ is a solution to the variational inequality

$$\mathit{find}\overline{u}\in K\mathit{such}\mathit{that}\langle \mathcal{F}\overline{u},u-\overline{u}\rangle \ge 0\forall u\in K$$

if and only if $\overline{u}$ is a solution to the following problem:

$$\mathit{find}\overline{u}\in K\mathit{such}\mathit{that}\langle \mathcal{F}u,u-\overline{u}\rangle \ge 0\forall u\in K.$$

Lemma 3

([23]). Let $\mathcal{H}$ be a real Hilbert space. Suppose that $\mathcal{F}:\mathcal{H}\to \mathcal{H}$ is κ-Lipschitzian and β-strongly monotone over a closed and convex subset $K\subset \mathcal{H}$. Then, the variational inequality problem

$$\mathit{find}\overline{u}\in K\mathit{such}\mathit{that}\langle \mathcal{F}\overline{u},v-\overline{u}\rangle \ge 0\forall v\in K$$

has a unique solution $\overline{u}\in K$.

Lemma 4

([24]). Let $\left\{{\mathrm{\Psi }}_n\right\}$ be a sequence of non-negative real numbers, $\left\{a_n\right\}$ be a sequence of real numbers in $(0,1)$ satisfying the condition ${\displaystyle \sum _{n=1}^{\infty }}{a}_n=\infty$, and $\left\{b_n\right\}$ be a sequence of real numbers. Assume that

$${\mathrm{\Psi }}_{n+1}\le (1-a_n){\mathrm{\Psi }}_n+a_n{b}_n,n\ge 1.$$

If $\underset{k\to \infty }{lim\; sup}{b}_{n_k}\le 0$ for every subsequence $\left\{{\mathrm{\Psi }}_{n_k}\right\}$ of $\left\{{\mathrm{\Psi }}_n\right\}$ satisfying $\underset{k\to \infty }{lim\; inf}\left({\mathrm{\Psi }}_{n_k+1}-{\mathrm{\Psi }}_{n_k}\right)\ge 0$, then $\underset{n\to \infty }{lim}{\mathrm{\Psi }}_n=0.$

3. Regularized Modified Forward-Backward Splitting Method

Let $S:\mathcal{H}\rightrightarrows \mathcal{H}$ be a maximal monotone operator, $T_i:\mathcal{H}\to \mathcal{H}$ be a monotone and $L_i$-Lipschitz continuous operator for $i\in \left[I\right]:=\{1,2,\cdots ,I\}$, and $a_i\in (0,1)$ such that ${\sum }_{i\in \left[I\right]}{a}_i=1$. We denote the solution set of problem (2) by $\mathrm{\Omega }$, that is,

$$\mathrm{\Omega }:=\{u\in \mathcal{H}:0\in (\sum _{i\in \left[I\right]}{a}_i{T}_i+S)u\},$$

and assume that $\mathrm{\Omega }\ne \varnothing$. Let $\mathcal{F}:\mathcal{H}\to \mathcal{H}$ be a $\gamma$-strongly monotone and L-Lipschitz continuous operator. We are seeking a solution $x^{*}\in \mathcal{H}$ such that

$$x^{*}\in \mathrm{\Omega }\mathrm{and}\langle \mathcal{F}{x}^{*},u^{†}-x^{*}\rangle \ge 0\forall u^{†}\in \mathrm{\Omega }.$$

(6)

We denote the solution set of (6) by $\tilde{\mathrm{\Omega }}$. In this connection, we consider the following regularized modified variational inclusion problem (RMVIP):

$$\mathrm{find}u\in \mathcal{H}\mathrm{such}\mathrm{that}0\in (\sum _{i\in \left[I\right]}{a}_i{T}_i+S)u+\tau \mathcal{F}u,$$

(7)

where $\tau >0$ is a regularization parameter. Observe that by Lemma 1, ${\sum }_{i\in \left[I\right]}{a}_i{T}_i+S$ is a maximal monotone operator. In addition, $({\sum }_{i\in \left[I\right]}{a}_i{T}_i+S)+\tau \mathcal{F}$ is strongly monotone because $\mathcal{F}$ is strongly monotone. Therefore, for each $\tau >0$, problem (7) possesses a unique solution which we denote by $u_{\tau }$. The following result concerning the solution net ${\left\{u_{\tau }\right\}}_{\tau \in (0,1)}$ can be deduced from [25] of Lemma 2, see also [7] of Proposition 3.1, but we will give the proof for completeness.

Proposition 1.

Let ${\left\{u_{\tau }\right\}}_{\tau \in (0,1)}$ be the solution net of (7). Then,

 (a)

${\left\{u_{\tau }\right\}}_{\tau \in (0,1)}$ is bounded.

 (b)

Let $u_{{\tau }_j}\in {\left\{u_{\tau }\right\}}_{\tau \in (0,1)}$, $j=1,2$. Then, $\parallel u_{{\tau }_1}-u_{{\tau }_2}\parallel \le \frac{|{\tau }_1-{\tau }_2|}{{\tau }_1}M$, where $M>0$ is a constant.

 (c)

$\underset{\tau \to 0^{+}}{lim}{u}_{\tau }$ exists and belongs to $\tilde{\mathrm{\Omega }}$.

Proof. 

(a) Let $u_{\tau }$ be a solution to (7). Then we have

$$-\tau \mathcal{F}{u}_{\tau }-\sum _{i\in \left[I\right]}{a}_i{T}_i{u}_{\tau }\in S{u}_{\tau }.$$

(8)

Choose $u^{†}\in \mathrm{\Omega }$. Then,

$$-\sum _{i\in \left[I\right]}{a}_i{T}_i{u}^{†}\in S{u}^{†}.$$

(9)

Using (8) and (9), we obtain

$$\langle -\tau \mathcal{F}{u}_{\tau }-\sum _{i\in \left[I\right]}{a}_i{T}_i{u}_{\tau }+\sum _{i\in \left[I\right]}{a}_i{T}_i{u}^{†},u_{\tau }-u^{†}\rangle \ge 0.$$

(10)

Since $T_i$ is monotone, it follows from (10) that

$$\tau \langle \mathcal{F}{u}_{\tau },u^{†}-u_{\tau }\rangle \ge \sum _{i\in \left[I\right]}{a}_i\langle T_i{u}_{\tau }-T_i{u}^{†},u_{\tau }-u^{†}\rangle \ge 0.$$

(11)

Using (11) and the $\gamma$-strong monotonicity of $\mathcal{F}$, we find that

$$\begin{array}{ccc}\hfill \gamma \parallel u^{†}-u_{\tau }{\parallel }^2& \le & \langle \mathcal{F}{u}^{†}-\mathcal{F}{u}_{\tau },u^{†}-u_{\tau }\rangle \hfill \\ & \le & \langle \mathcal{F}{u}^{†},u^{†}-u_{\tau }\rangle \hfill \\ & \le & \parallel \mathcal{F}{u}^{†}\parallel \parallel u^{†}-u_{\tau }\parallel .\hfill \end{array}$$

(12)

It now follows from (12) that $\parallel u^{†}-u_{\tau }\parallel \le \frac{\parallel \mathcal{F}{u}^{†}\parallel }{\gamma }$ and hence ${\left\{u_{\tau }\right\}}_{\tau \in (0,1)}$ is bounded.

(b) Let $u_{{\tau }_j}\in {\left\{u_{\tau }\right\}}_{\tau \in (0,1)}$, $j=1,2$. Then,

$$0\in (\sum _{i\in \left[I\right]}{a}_i{T}_i+S)u_{{\tau }_j}+{\tau }_j\mathcal{F}{u}_{{\tau }_j}.$$

(13)

Using the monotonicity of $({\sum }_{i\in \left[I\right]}{a}_i{T}_i+S)$ and (13), we find that

$$\langle -{\tau }_1\mathcal{F}{u}_{{\tau }_1}+{\tau }_2\mathcal{F}{u}_{{\tau }_1}-{\tau }_2\mathcal{F}{u}_{{\tau }_1}+{\tau }_2\mathcal{F}{u}_{{\tau }_2},u_{{\tau }_1}-u_{{\tau }_2}\rangle =\langle -{\tau }_1\mathcal{F}{u}_{{\tau }_1}+{\tau }_2\mathcal{F}{u}_{{\tau }_2},u_{{\tau }_1}-u_{{\tau }_2}\rangle \ge 0.$$

(14)

Using (14), we see that

$$({\tau }_2-{\tau }_1)\langle \mathcal{F}{u}_{{\tau }_1},u_{{\tau }_1}-u_{{\tau }_2}\rangle \ge {\tau }_2\langle \mathcal{F}{u}_{{\tau }_1}-\mathcal{F}{u}_{{\tau }_2},u_{{\tau }_1}-u_{{\tau }_2}\rangle \ge {\tau }_2\gamma {\parallel u_{{\tau }_1}-u_{{\tau }_2}\parallel }^2.$$

(15)

Therefore, it follows from (15) that

$$\begin{array}{ccc}\hfill {\tau }_2\gamma {\parallel u_{{\tau }_1}-u_{{\tau }_2}\parallel }^2& \le & ({\tau }_2-{\tau }_1)\langle \mathcal{F}{u}_{{\tau }_1},u_{{\tau }_1}-u_{{\tau }_2}\rangle \hfill \\ & \le & |{\tau }_2-{\tau }_1|\parallel \mathcal{F}{u}_{{\tau }_1}\parallel \parallel u_{{\tau }_1}-u_{{\tau }_2}\parallel .\hfill \end{array}$$

(16)

Using (16), we obtain

$$\parallel u_{{\tau }_1}-u_{{\tau }_2}\parallel \le \frac{|{\tau }_2-{\tau }_1|\parallel \mathcal{F}{u}_{{\tau }_1}\parallel }{{\tau }_2\gamma }=\frac{|{\tau }_2-{\tau }_1|M}{{\tau }_2},$$

where $M=\frac{\parallel \mathcal{F}{u}_{{\tau }_1}\parallel }{\gamma }$.

(c) The boundedness of the net ${\left\{u_{\tau }\right\}}_{\tau \in (0,1)}$ implies that there exists a subsequence $\left\{u_{{\tau }_n}\right\}$ of $\left\{u_{\tau }\right\}$ such that $u_{{\tau }_n}\rightharpoonup \overline{u}$ as $n\to \infty$ where ${\left({\tau }_n\right)}_{n\in \mathbb{N}}$ is a sequence in $(0,1)$ such that ${\tau }_n\to 0^{+}$ as $n\to \infty$. Note that $-{\tau }_n\mathcal{F}{u}_{{\tau }_n}\in ({\sum }_{i\in \left[I\right]}{a}_i{T}_i+S)u_{{\tau }_n}$ for each n. Since the operator $({\sum }_{i\in \left[I\right]}{a}_i{T}_i+S)$ is maximal monotone and hence sequentially closed in the weak-strong topology on $\mathcal{H}\times \mathcal{H}$, taking $n\to \infty$, we infer that $0\in ({\sum }_{i\in \left[I\right]}{a}_i{T}_i+S)\overline{u}$. Therefore, $\overline{u}\in \mathrm{\Omega }$. In addition, $0\in ({\sum }_{i\in \left[I\right]}{a}_i{T}_i+S)x^{*}$ and $\langle \mathcal{F}{x}^{*},u^{†}-x^{*}\rangle \ge 0\forall u^{†}\in \mathrm{\Omega }$. It then follows that

$$\langle -{\tau }_n\mathcal{F}{u}_{{\tau }_n}-\sum _{i\in \left[I\right]}{a}_i{T}_i{u}_{{\tau }_n}+\sum _{i\in \left[I\right]}{a}_i{T}_i{x}^{*},u_{{\tau }_n}-x^{*}\rangle \ge 0.$$

(17)

It now follows from (17) and the monotonicity of $T_i$ that

$$-\langle {\tau }_n\mathcal{F}{u}_{{\tau }_n},u_{{\tau }_n}-x^{*}\rangle \ge \sum _{i\in \left[I\right]}{a}_i\langle T_i{u}_{{\tau }_n}-T_i{x}^{*},u_{{\tau }_n}-x^{*}\rangle \ge 0.$$

(18)

Using (18), we conclude that

$$\langle \mathcal{F}{u}_{{\tau }_n}-\mathcal{F}{x}^{*},u_{{\tau }_n}-x^{*}\rangle +\langle \mathcal{F}{x}^{*},u_{{\tau }_n}-x^{*}\rangle =\langle \mathcal{F}{u}_{{\tau }_n},u_{{\tau }_n}-x^{*}\rangle \le 0.$$

(19)

Moreover, using (19) and the $\gamma$-strong monotonicity of $\mathcal{F}$, we see that

$$\gamma \parallel u_{{\tau }_n}-x^{*}{\parallel }^2\le \langle \mathcal{F}{u}_{{\tau }_n}-\mathcal{F}{x}^{*},u_{{\tau }_n}-x^{*}\rangle \le \langle \mathcal{F}{x}^{*},x^{*}-u_{{\tau }_n}\rangle .$$

(20)

Therefore, using (20), we have

$$\langle \mathcal{F}{x}^{*},x^{*}-u_{{\tau }_n}\rangle \ge 0.$$

(21)

Passing to the limit in (21) as $n\to \infty$, we find that

$$\langle \mathcal{F}{x}^{*},x^{*}-\overline{u}\rangle \ge 0\forall x^{*}\in \mathrm{\Omega },\overline{u}\in \mathrm{\Omega },$$

and using Lemma 2, we arrive at the conclusion that $\overline{u}$ solves (6). The uniqueness of the solution to $\langle \mathcal{F}{x}^{*},u^{†}-x^{*}\rangle \ge 0$ according to Lemma 3 implies that $\overline{u}=x^{*}$. Thus, passing to the limit as $n\to \infty$ in (20), we conclude that $u_{{\tau }_n}\to x^{*}$ as $n\to \infty$.    □

Following an approach similar to the above analysis, we denote the solution set of problem (4) by ${\mathrm{\Omega }}_2$ and consider the corresponding regularized common variational inclusion problem:

$$\mathrm{find}u\in \mathcal{H}\mathrm{such}\mathrm{that}0\in (T_i+S_i)u+\tau \mathcal{F}u\mathrm{for}\mathrm{all}i.$$

(22)

If ${\left\{u_{\tau }\right\}}_{\tau \in (0,1)}$ is the solution net of (22), then Proposition 1 (a) and (b) hold, and for (c), we see that ${lim}_{\tau \to 0^{+}}{u}_{\tau }$ exists and belongs to ${\tilde{\mathrm{\Omega }}}_2:=\{x^{*}\in {\mathrm{\Omega }}_2:\langle \mathcal{F}{x}^{*},u^{*}-x^{*}\rangle \ge 0\forall u^{*}\in {\mathrm{\Omega }}_2\}$.

We make the following assumptions in connection with our Algorithm 1.

Assumption 1.

(A1) ${\tau }_n\in (0,1)$, $\underset{n\to \infty }{lim}{\tau }_n=0$, and ${\displaystyle \sum _{n=1}^{\infty }}{\tau }_n=\infty$;

 (A2)

$\underset{n\to \infty }{lim}\frac{|{\tau }_{n+1}-{\tau }_n|}{{\tau }_n^2}=0$;

 (A3)

$\left\{{\mu }_n\right\}$ and $\left\{p_n\right\}$ are two real sequences satisfying ${\displaystyle \sum _{n=1}^{\infty }}{\mu }_n<\infty$ and ${\displaystyle \sum _{n=1}^{\infty }}{p}_n<\infty$, respectively.

We next present a unified algorithm for solving the aforementioned problems.

Algorithm 1: Regularized Modified Forward-Backward Splitting Method (RMFBSM)

Initialization: Let $u_1\in \mathcal{H}$, $\mu \in (0,1)$, and ${\lambda }_1>0$ be given.

Iterative steps: Given $u_n$, calculate $u_{n+1}$ as follows:

Step 1: Compute $$y_n^i=J_{{\lambda }_n{S}_i}(u_n-{\lambda }_n{T}_i{u}_n-{\lambda }_n{\tau }_n\mathcal{F}{u}_n).$$ (23)

Step 2: Find $i_n:=arg{max}_{i\in \left[I\right]}\{\parallel y_n^i-u_n\parallel \}$.

Step 3: Compute $$u_{n+1}=y_n^{i_n}-{\lambda }_n(T_{i_n}{y}_n^{i_n}-T_{i_n}{u}_n).$$ (24)

Update $$\begin{array}{c}\hfill {\lambda }_{n+1}=\left\{\begin{array}{cc}min\left\{{\lambda }_n+{\rho }_n,\frac{(\mu +{\mu }_n)\parallel y_n-u_n\parallel }{\parallel T_{i_n}{y}_n^{i_n}-T_{i_n}{u}_n\parallel }\right\},\hfill & if{T}_{i_n}{y}_n^{i_n}\ne T_{i_n}{u}_n,\hfill \\ {\lambda }_n+{\rho }_n,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.\end{array}$$

Set $n:=n+1$ and go back to Step 1.

Remark 2.

If $T_i=T$, $S_i=S$, ${\rho }_n=0={\mu }_n$ in Algorithm 1, and $d_n=0$ in Algorithm 1 of Hieu et al. [25], then the two algorithms are the same. However, the problems we intend to solve are more general than the problem (VIP) studied by Hieu et al. The sequence of step sizes in Algorithm 1 is convergent, as shown in the next lemma.

Lemma 5.

The sequence $\left\{{\lambda }_n\right\}$ generated by Algorithm 1 is bounded, and ${\lambda }_n\in [min\{\frac{\mu }{L_{i_n}},{\lambda }_1\},{\lambda }_1+P]$. Moreover, there exists $\lambda \in [min\{\frac{\mu }{L_{i_n}},{\lambda }_1\},{\lambda }_1+P]$ such that $\underset{n\to \infty }{lim}{\lambda }_n=\lambda$, where $P={\displaystyle \sum _{n=1}^{\infty }}{\rho }_n$.

Proof. 

The proof follows from [26] of Lemma 2.1.    □

Lemma 6.

Let $\left\{u_n\right\}$ be the sequence generated by Algorithm 1. Then, there exists $n_0\in \mathbb{N}$ such that

$$\parallel u_{n+1}-u_{{\tau }_n}{\parallel }^2\le \left(1-\gamma {\lambda }_n{\tau }_n\right)\parallel u_n-u_{{\tau }_n}{\parallel }^2-(1-\mu ){\parallel y_n^i-u_n\parallel }^2\forall i\in \left[I\right],n\ge n_0,$$

where $u_{{\tau }_n}$ solves the RMVIP with $u_{\tau }$ replaced by $u_{{\tau }_n}$.

Proof. 

Using the definition of $y_n^i$, we obtain

$$u_n-{\lambda }_n{T}_i{u}_n-{\lambda }_n{\tau }_n\mathcal{F}{u}_n\in y_n^i+{\lambda }_n{S}_i{y}_n^i,$$

from which it follows that

$$u_n-{\lambda }_n(T_i{u}_n-T_i{y}_n^i)-{\lambda }_n{\tau }_n\mathcal{F}{u}_n-y_n^i\in {\lambda }_n(T_i+S_i)y_n^i,\mathrm{for}\mathrm{each}i.$$

(25)

Employing (24) and (25) in particular for $i_n$, we obtain

$$u_n-u_{n+1}-{\lambda }_n{\tau }_n\mathcal{F}{u}_n\in {\lambda }_n(T_{i_n}+S_{i_n})y_n^{i_n}.$$

(26)

In addition,

$$-{\lambda }_n{\tau }_n\mathcal{F}{u}_{{\tau }_n}\in {\lambda }_n(T_{i_n}+S_{i_n})u_{{\tau }_n}.$$

(27)

Therefore, it follows from (26) and (27) that

$$\begin{array}{c}\hfill \langle u_n-u_{n+1},y_n^{i_n}-u_{{\tau }_n}\rangle -{\lambda }_n{\tau }_n\langle \mathcal{F}{u}_n-\mathcal{F}{u}_{{\tau }_n},y_n^{i_n}-u_{{\tau }_n}\rangle \ge 0.\end{array}$$

(28)

Using the fact that $\mathcal{F}$ is $\gamma$-strongly monotone, we infer from (28) that

$$\begin{array}{ccc}\hfill \langle u_n-y_n^{i_n},y_n^{i_n}-u_{{\tau }_n}\rangle -\langle u_{n+1}-y_n^{i_n},y_n^{i_n}-u_{{\tau }_n}\rangle -{\lambda }_n{\tau }_n\langle \mathcal{F}{u}_n-\mathcal{F}{u}_{{\tau }_n},y_n^{i_n}-u_n\rangle & \ge & {\lambda }_n{\tau }_n\langle \mathcal{F}{u}_n-\mathcal{F}{u}_{{\tau }_n},u_n-u_{{\tau }_n}\rangle \hfill \\ & \ge & {\lambda }_n{\tau }_n\gamma {\parallel u_n-u_{{\tau }_n}\parallel }^2.\hfill \end{array}$$

(29)

Note that

$$\langle u_n-y_n^{i_n},y_n^{i_n}-u_{{\tau }_n}\rangle =\frac{1}{2}\left(\parallel u_n-u_{{\tau }_n}{\parallel }^2-\parallel u_n-y_n^{i_n}{\parallel }^2-{\parallel y_n^{i_n}-u_{{\tau }_n}\parallel }^2\right)$$

(30)

and

$$\langle u_{n+1}-y_n^{i_n},y_n^{i_n}-u_{{\tau }_n}\rangle =\frac{1}{2}\left(\parallel u_{n+1}-u_{{\tau }_n}{\parallel }^2-\parallel u_{n+1}-y_n^{i_n}{\parallel }^2-{\parallel y_n^{i_n}-u_{{\tau }_n}\parallel }^2\right).$$

(31)

Substituting (30) and (31) in (29) and multiplying throughout by 2, we obtain

$$\begin{array}{ccc}\hfill \parallel u_{n+1}-u_{{\tau }_n}{\parallel }^2& \le & \parallel u_n-u_{{\tau }_n}{\parallel }^2-\parallel u_n-y_n^{i_n}{\parallel }^2+{\parallel u_{n+1}-y_n^{i_n}\parallel }^2-2{\lambda }_n{\tau }_n\langle \mathcal{F}{u}_n-\mathcal{F}{u}_{{\tau }_n},y_n^{i_n}-u_n\rangle \hfill \\ & & -2{\lambda }_n{\tau }_n\gamma {\parallel u_n-u_{{\tau }_n}\parallel }^2\hfill \\ & =& (1-2{\lambda }_n{\tau }_n\gamma ){\parallel u_n-u_{{\tau }_n}\parallel }^2-2{\lambda }_n{\tau }_n\langle \mathcal{F}{u}_n-\mathcal{F}{u}_{{\tau }_n},y_n^{i_n}-u_n\rangle \hfill \\ & & -\parallel u_n-y_n^{i_n}{\parallel }^2+{\parallel u_{n+1}-y_n^{i_n}\parallel }^2.\hfill \end{array}$$

(32)

Using (24) and the Peter-Paul inequality, as well as (32), for any ${ϵ}_1>0$, we have

$$\begin{array}{ccc}\hfill \parallel u_{n+1}-u_{{\tau }_n}{\parallel }^2& \le & (1-2{\lambda }_n{\tau }_n\gamma )\parallel u_n-u_{{\tau }_n}{\parallel }^2+\frac{{\lambda }_n{\tau }_{n}L}{{ϵ}_1}\parallel u_n-u_{{\tau }_n}{\parallel }^2+{\lambda }_n{\tau }_{n}L{ϵ}_1{\parallel y_n^{i_n}-u_n\parallel }^2\hfill \\ & & -\parallel u_n-y_n^{i_n}{\parallel }^2+{\lambda }_n^2{\parallel T_{i_n}{y}_n^{i_n}-T_{i_n}{u}_n\parallel }^2\hfill \\ & \le & \left(1-{\lambda }_n{\tau }_n(2\gamma -\frac{L}{{ϵ}_1})\right)\parallel u_n-u_{{\tau }_n}{\parallel }^2-\left(1-{\lambda }_n{\tau }_{n}L{ϵ}_1-\frac{{\lambda }_n^2{(\mu +{\mu }_n)}^2}{{\lambda }_{n+1}^2}\right){\parallel y_n^{i_n}-u_n\parallel }^2.\hfill \end{array}$$

(33)

Choose ${ϵ}_1=\frac{L}{\gamma }$. Observe that from Lemma 5 and the Assumption 1 (A1) and Assumption 1 (A3),

$$\underset{n\to \infty }{lim}\left(1-\frac{{\lambda }_n{\tau }_n{L}^2}{\gamma }-\frac{{\lambda }_n^2{(\mu +{\mu }_n)}^2}{{\lambda }_{n+1}^2}\right)=1-{\mu }^2>1-\mu >0.$$

Therefore, there exists $n_0\in \mathbb{N}$ such that $\left(1-\frac{{\lambda }_n{\tau }_n{L}_1^2}{\gamma }-\frac{{\lambda }_n^2{\mu }^2}{{\lambda }_{n+1}^2}\right)>(1-\mu )$ for all $n\ge n_0$. Thus, it follows from (33) that

$$\begin{array}{ccc}\hfill \parallel u_{n+1}-u_{{\tau }_n}{\parallel }^2& \le & (1-\gamma {\lambda }_n{\tau }_n)\parallel u_n-u_{{\tau }_n}{\parallel }^2-(1-\mu ){\parallel y_n^{i_n}-u_n\parallel }^2\hfill \\ & \le & (1-\gamma {\lambda }_n{\tau }_n)\parallel u_n-u_{{\tau }_n}{\parallel }^2-(1-\mu ){\parallel y_n^i-u_n\parallel }^2,\forall i\in \left[I\right],\forall n\ge n_0,\hfill \end{array}$$

(34)

as asserted.    □

Theorem 1.

The sequences generated by Algorithm 1 converge strongly to $x^{*}\in \widetilde{{\mathrm{\Omega }}_2}$.

Proof. 

Invoking Proposition 1(b) and the Peter-Paul inequality, we see that there exists ${ϵ}_2>0$ such that

$$\begin{array}{ccc}\hfill \parallel u_{n+1}-u_{{\tau }_n}{\parallel }^2& =& \parallel u_{n+1}-u_{{\tau }_{n+1}}+u_{{\tau }_{n+1}}-u_{{\tau }_n}{\parallel }^2\hfill \\ & =& \parallel u_{n+1}-u_{{\tau }_{n+1}}{\parallel }^2+2\langle u_{n+1}-u_{{\tau }_{n+1}},u_{{\tau }_{n+1}}-u_{{\tau }_n}\rangle +{\parallel u_{{\tau }_{n+1}}-u_{{\tau }_n}\parallel }^2\hfill \\ & \ge & \parallel u_{n+1}-u_{{\tau }_{n+1}}{\parallel }^2-{ϵ}_2\parallel u_{n+1}-u_{{\tau }_{n+1}}{\parallel }^2-\frac{\parallel u_{{\tau }_{n+1}}-u_{{\tau }_n}{\parallel }^2}{{ϵ}_2}+{\parallel u_{{\tau }_{n+1}}-u_{{\tau }_n}\parallel }^2\hfill \\ & =& (1-{ϵ}_2)\parallel u_{n+1}-u_{{\tau }_{n+1}}{\parallel }^2-\left(\frac{1-{ϵ}_2}{{ϵ}_2}\right){\parallel u_{{\tau }_{n+1}}-u_{{\tau }_n}\parallel }^2\hfill \\ & \ge & (1-{ϵ}_2){\parallel u_{n+1}-u_{{\tau }_{n+1}}\parallel }^2-\left(\frac{1-{ϵ}_2}{{ϵ}_2}\right){\left(\frac{{\tau }_{n+1}-{\tau }_n}{{\tau }_n}\right)}^2{M}^2.\hfill \end{array}$$

(35)

Next, we define the sequence $\left\{{\beta }_n\right\}$ by

$$\begin{array}{c}\hfill {\beta }_n:=\left\{\begin{array}{cc}{\tau }_n,\hfill & \mathrm{if}\gamma {\lambda }_n>1\hfill \\ \gamma {\lambda }_n{\tau }_n,\hfill & \mathrm{if}\gamma {\lambda }_n\le 1.\hfill \end{array}\right.\end{array}$$

Then, from Lemma 6 and (35), it follows that $\forall i\in \left[I\right]$ and for $n\ge n_0$,

$$\begin{array}{ccc}\hfill \parallel u_{n+1}-u_{{\tau }_{n+1}}{\parallel }^2& \le & \frac{(1-{\beta }_n)}{1-{ϵ}_2}{\parallel u_n-u_{{\tau }_n}\parallel }^2+\frac{1}{{ϵ}_2}{\left(\frac{{\tau }_{n+1}-{\tau }_n}{{\tau }_n}\right)}^2{M}^2\hfill \\ & & -\frac{1}{1-{ϵ}_2}(1-\mu ){\parallel y_n^i-u_n\parallel }^2.\hfill \end{array}$$

(36)

Let ${ϵ}_2=0.5{\beta }_n$. Then, from (36), it follows that

$$\begin{array}{ccc}\hfill \parallel u_{n+1}-u_{{\tau }_{n+1}}{\parallel }^2& \le & (1-\frac{{ϵ}_2}{1-{ϵ}_2}){\parallel u_n-u_{{\tau }_n}\parallel }^2+\frac{{ϵ}_2}{1-{ϵ}_2}\left(\frac{1-{ϵ}_2}{{ϵ}_2^2}{\left(\frac{{\tau }_{n+1}-{\tau }_n}{{\tau }_n}\right)}^2{M}^2\right)\hfill \\ & & -\frac{1-\mu }{1-{ϵ}_2}{\parallel y_n^i-u_n\parallel }^2\hfill \\ & =& (1-\frac{{ϵ}_2}{1-{ϵ}_2}){\parallel u_n-u_{{\tau }_n}\parallel }^2+\frac{{ϵ}_2}{1-{ϵ}_2}\left(\frac{4(1-{ϵ}_2)}{{\beta }_n^2}{\left(\frac{{\tau }_{n+1}-{\tau }_n}{{\tau }_n}\right)}^2{M}^2\right)\hfill \\ & & -\frac{1-\mu }{1-{ϵ}_2}{\parallel y_n^i-u_n\parallel }^2,\forall n\ge n_0.\hfill \end{array}$$

(37)

Let ${\mathrm{\Theta }}_n={\parallel u_n-u_{{\tau }_n}\parallel }^2$, ${\delta }_n=\frac{{ϵ}_2}{1-{ϵ}_2}$ and

$$b_n=\frac{4(1-{ϵ}_2)}{{\beta }_n^2}{\left(\frac{{\tau }_{n+1}-{\tau }_n}{{\tau }_n}\right)}^2{M}^2.$$

We see that ${\delta }_n\in (0,1)$ and ${\displaystyle \sum _{n=1}^{\infty }}{\delta }_n=\infty$. Then, from (37), we obtain

$$\begin{array}{ccc}\hfill {\mathrm{\Theta }}_{n+1}& \le & (1-{\delta }_n){\mathrm{\Theta }}_n+{\delta }_n{b}_n-\frac{1-\mu }{1-{ϵ}_2}{\parallel y_n^i-u_n\parallel }^2,\forall n\ge n_0.\hfill \end{array}$$

(38)

It is not difficult to see that the sequence $\left\{b_n\right\}$ is bounded. To complete the proof, we invoke Lemma 4 by showing that $\underset{k\to \infty }{lim\; sup}{b}_{n_k}\le 0$ for every subsequence $\left\{{\mathrm{\Theta }}_{n_k}\right\}$ of $\left\{{\mathrm{\Theta }}_n\right\}$ that satisfies $\underset{k\to \infty }{lim\; inf}({\mathrm{\Theta }}_{n_k+1}-{\mathrm{\Theta }}_{n_k})\ge 0$. Therefore, let $\left\{{\mathrm{\Theta }}_{n_k}\right\}$ be a subsequence of $\left\{{\mathrm{\Theta }}_n\right\}$ such that $\underset{k\to \infty }{lim\; inf}({\mathrm{\Theta }}_{n_k+1}-{\mathrm{\Theta }}_{n_k})\ge 0$. It then follows from (38) that

$$\begin{array}{ccc}\hfill \underset{k\to \infty }{lim\; sup}\frac{1-\mu }{(1-{ϵ}_2)}{\parallel y_{n_k}^i-u_{n_k}\parallel }^2& \le & \underset{k\to \infty }{lim\; sup}\left(({\mathrm{\Theta }}_{n_k}-{\mathrm{\Theta }}_{n_k+1})+{\delta }_{n_k}(b_{n_k}-{\mathrm{\Theta }}_{n_k})\right)\hfill \\ & \le & -\underset{k\to \infty }{lim\; inf}({\mathrm{\Theta }}_{n_k+1}-{\mathrm{\Theta }}_{n_k})\hfill \\ & \le & 0.\hfill \end{array}$$

(39)

From (39), we infer that

$$\underset{k\to \infty }{lim}\parallel y_{n_k}^i-u_{n_k}\parallel =0.$$

(40)

Moreover, from (24) and (40), it follows that

$$\underset{k\to \infty }{lim}\parallel u_{n_k+1}-u_{n_k}\parallel =0.$$

Indeed, it immediately follows from conditions (A1) and (A2) that $\underset{k\to \infty }{lim\; sup}{b}_k=0$. Therefore, using Lemma 4, we obtain that $\underset{n\to \infty }{lim}{\mathrm{\Theta }}_n=0$. Consequently, using Proposition 1 (c) and (40), we conclude that $\underset{n\to \infty }{lim}{u}_n=\underset{n\to \infty }{lim}{y}_n^i=x^{*}$.    □

We next present some consequences of the above result to confirm that our Algorithm 1 indeed provides a unified framework for solving the inclusion problems (2)–(4).

Corollary 1.

Let $S:\mathcal{H}\rightrightarrows \mathcal{H}$ be a maximal monotone operator and $T_i:\mathcal{H}\to \mathcal{H}$ be monotone Lipschitz continuous operators with constants $L_i$ for $i\in \left[I\right]$. Let $\mathcal{F}:\mathcal{H}\to \mathcal{H}$ be a γ-strongly monotone and Lipschitz continuous operator with constant L. Assume ${\mathrm{\Omega }}_1:=\{x\in \mathcal{H}:0\in {\cap }_{i\in \left[I\right]}(T_i+S)\}\ne \varnothing$. Then, the sequences generated by Algorithm 2, under Assumption 1, converge strongly to $x^{*}\in {\mathrm{\Omega }}_1$, satisfying $\langle \mathcal{F}{x}^{*},u^{*}-x^{*}\rangle \ge 0\forall u^{*}\in {\mathrm{\Omega }}_1$.

Algorithm 2: Regularized Modified Forward-Backward Splitting Method (RMFBSM)

Initialization: Let $u_1\in \mathcal{H}$, $\mu \in (0,1)$, and ${\lambda }_1>0$ be given.

Iterative steps: Calculate $u_{n+1}$ as follows:

Step 1: Compute $$y_n^i=J_{{\lambda }_{n}S}(u_n-{\lambda }_n{T}_i{u}_n-{\lambda }_n{\tau }_n\mathcal{F}{u}_n).$$

Step 2: Find $i_n:=arg{max}_{i\in \left[I\right]}\{\parallel y_n^i-u_n\parallel \}$.

Step 3: Compute $$u_{n+1}=y_n^{i_n}-{\lambda }_n(T_{i_n}{y}_n^{i_n}-T_{i_n}{u}_n).$$

Update $$\begin{array}{c}\hfill {\lambda }_{n+1}=\left\{\begin{array}{cc}min\left\{{\lambda }_n+{\rho }_n,\frac{(\mu +{\mu }_n)\parallel y_n-u_n\parallel }{\parallel T_{i_n}{y}_n^{i_n}-T_{i_n}{u}_n\parallel }\right\},\hfill & if{T}_{i_n}{y}_n^{i_n}\ne T_{i_n}{u}_n,\hfill \\ {\lambda }_n+{\rho }_n,\hfill & otherwise.\hfill \end{array}\right.\end{array}$$

Set $n:=n+1$ and go back to Step 1.

Proof. 

Note that Algorithm 2 is derived from Algorithm 1 by taking $S_i=S$ for $i\in \left[I\right]$. Thus, the proof follows from the proof of Theorem 1.    □

Corollary 2.

Let $S:\mathcal{H}\rightrightarrows \mathcal{H}$ be a maximal monotone operator and $T_j:\mathcal{H}\to \mathcal{H}$ be monotone Lipschitz continuous operators with constants $L_j$ for $j\in \left[J\right]$. Let $\mathcal{F}:\mathcal{H}\to \mathcal{H}$ be a γ-strongly monotone and Lipschitz continuous operator with constant L and $a_j\in (0,1)$ such that ${\displaystyle \sum _{j\in \left[J\right]}^I}{a}_j=1$. Assume that $\mathrm{\Omega }:=\{x\in \mathcal{H}:0\in {\sum }_{j\in \left[J\right]}(a_j{T}_j+S)\}\ne \varnothing$. Then, the sequences generated by Algorithm 3, under Assumption 1, converge strongly to $x^{*}\in \mathrm{\Omega }$, satisfying $\langle \mathcal{F}{x}^{*},u^{*}-x^{*}\rangle \ge 0\forall u^{*}\in \mathrm{\Omega }$.

Algorithm 3: Regularized Modified Forward-Backward Splitting Method (RMFBSM)

Initialization: Let $u_1\in \mathcal{H}$, $\mu \in (0,1)$, and ${\lambda }_1>0$ be given.

Iterative steps:  Given $u_n$, calculate $u_{n+1}$ as follows:

Step 1: Compute $$y_n=J_{{\lambda }_{n}S}(u_n-{\lambda }_n\sum _{j=1}^J{a}_j{T}_j{u}_n-{\lambda }_n{\tau }_n\mathcal{F}{u}_n).$$

Step 2: Compute $$u_{n+1}=y_n-{\lambda }_n\sum _{i=1}^J{a}_j(T_j{y}_n-T_j{u}_n).$$ (41)

Update $$\begin{array}{c}\hfill {\lambda }_{n+1}=\left\{\begin{array}{cc}min\left\{{\lambda }_n+{\rho }_n,\frac{(\mu +{\mu }_n)\parallel y_n-u_n\parallel }{\parallel {\sum }_{j\in \left[J\right]}{a}_j(T_j{y}_n-T_j{u}_n)\parallel }\right\},\hfill & if\sum _{j\in \left[J\right]}{a}_j{T}_j{y}_n\ne \sum _{j\in \left[J\right]}{a}_j{T}_j{u}_n,\hfill \\ {\lambda }_n+{\rho }_n,\hfill & otherwise.\hfill \end{array}\right.\end{array}$$

Set $n:=n+1$ and go back to Step 1.

Proof. 

Note that Algorithm 3 is derived from Algorithm 1 by taking $I=1$ and setting $T_1=T={\sum }_{j\in \left[J\right]}{a}_j{T}_j$. Note that the new operator T is monotone and Lipschitz continuous. Thus, the proof follows from the proof of Theorem 1.    □

Remark 3.

On the other hand, we note that the structure of some steps in Algorithm 1 differs from that of Algorithm 1 of [18]. For instance, the authors of [18] needed to solve an optimization problem to compute $u_{n+1}$. Our iterative scheme 3 provides an alternative and bypasses such difficulties. However, we do not consider accelerated schemes in the present study because such results can easily be obtained from the results in a recent work of ours [27].

4. Applications

In this section, in the cases where $I=1$, we will consider two applications of our main results.

4.1. Split Feasibility Problems

Let C and Q be nonempty, closed, and convex subsets of the real Hilbert spaces ${\mathcal{H}}_1$ and ${\mathcal{H}}_2$, respectively, and let $A:{\mathcal{H}}_1\to {\mathcal{H}}_2$ be a bounded linear operator, the adjoint of which is denoted by $A^{*}:{\mathcal{H}}_2\to {\mathcal{H}}_1$. We now recall the split feasibility problems (SFP):

$$\mathrm{find}u\in C\mathrm{such}\mathrm{that}Au\in Q.$$

(42)

We denote the set of solutions of (42) by $\mathcal{S}$ and assume that $\mathcal{S}\ne \varnothing$. The concept of SFP was introduced by Censor and Elfving [28] in 1994 in the framework of Euclidean spaces. It has been successfully applied to model some inverse problems in medical image reconstruction, phase retrievals, gene regulatory network inference, and intensity modulation radiation therapy; see, for example, [29,30]. From its conception to date, the SFP have been widely studied by several authors who also proposed various iterative schemes for solving them. Interested readers should consult [31,32,33,34,35,36,37,38] and references therein for recent studies and generalizations of this problem.

One can verify that the solution set of SFP (42) coincides with the solution set of the following constrained minimization problem; see, for example, [36]:

$$\underset{u\in C}{min}f\left(u\right):=\frac{1}{2}{\parallel Au-P_{Q}Au\parallel }^2.$$

(43)

However, it is to be observed that the minimization problem (43) is ill-posed, in general, and therefore calls for regularization. We consider the following Tikhonov regularization [38]:

$$\underset{u\in C}{min}{f}^{\kappa }\left(u\right):=\frac{1}{2}\parallel Au-P_Q{Au\parallel }^2+\frac{1}{2}\kappa {\parallel u\parallel }^2.$$

(44)

Equivalently, (44) can be written as the following unconstrained minimization problem:

$$\underset{u\in {\mathcal{H}}_1}{min}{F}^{\kappa }\left(u\right):=\frac{1}{2}\parallel Au-P_Q{Au\parallel }^2+\frac{1}{2}\kappa {\parallel u\parallel }^2+i_C\left(u\right).$$

(45)

Note that the function $f^{\kappa }$ is differentiable and its gradient $\nabla f^{\kappa }\left(u\right)=A^{*}(Au-P_{Q}Au)+\kappa u$. Problem (45) is equivalent to the following inclusion problem:

$$\mathrm{find}u\in {\mathcal{H}}_1\mathrm{such}\mathrm{that}0\in \nabla f^{\kappa }\left(u\right)+\partial i_C\left(u\right).$$

(46)

It is not difficult to see that $\nabla f^{\kappa }$ is monotone and $(\parallel A{\parallel }^2+\kappa )$ Lipschitz continuous. Therefore, since $\partial i_C$ is a maximal monotone operator, (46) assures us that we can apply our result to solving the SFP (42). Thence, our next result.

Theorem 2.

Let C and Q be nonempty, closed, and convex subsets of the real Hilbert spaces ${\mathcal{H}}_1$ and ${\mathcal{H}}_2$, respectively. Suppose that $A:{\mathcal{H}}_1\to {\mathcal{H}}_2$ is a bounded linear operator and let the operator $A^{*}:{\mathcal{H}}_2\to {\mathcal{H}}_1$ be its adjoint operator. Assume that $\mathcal{S}:=\{u\in {\mathcal{H}}_1:u\in CandA{u}^{*}\in Q\}\ne \varnothing$ and that Assumption 1 holds. Then, the sequence $\left\{u_n\right\}$ generated by Algorithm 4 converges strongly to a point $u^{*}\in \mathcal{S}$, satisfying $\langle \mathcal{F}{u}^{*},y-u^{*}\rangle \ge 0\forall y\in \mathcal{S}$.

Algorithm 4: RMFBSM Method for Solving SFP

Initialization: Let $u_1\in {\mathcal{H}}_1$, $\mu \in (0,1)$, and ${\lambda }_1>0$ be given. Iterative steps:

Calculate $u_{n+1}$ as follows: Step 1: Compute $$y_n=P_C(u_n-{\lambda }_n\nabla f^{\kappa }{u}_n-{\lambda }_n{\tau }_n\mathcal{F}{u}_n).$$

Step 2: Compute $$u_{n+1}=y_n-{\lambda }_n(\nabla f^{\kappa }{y}_n-\nabla f^{\kappa }{u}_n).$$

Update $$\begin{array}{c}\hfill {\lambda }_{n+1}=\left\{\begin{array}{cc}min\left\{{\lambda }_n,\frac{\mu \parallel y_n-u_n\parallel }{\parallel \nabla f^{\kappa }{y}_n-\nabla f^{\kappa }{u}_n\parallel }\right\},\hfill & \mathrm{if}\nabla f^{\kappa }{y}_n\ne \nabla f^{\kappa }{u}_n,\hfill \\ {\lambda }_n,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.\end{array}$$

Set $n:=n+1$ and go back to Step 1.

Proof. 

Let $S=\partial i_C$ and $T_i=T=\nabla f^{\kappa }$ for all $i=1,2,\cdots ,N$ in Algorithm 1. Note that by (5), we have $J_{{\lambda }_{n}S}=P_C$. Therefore, the proof can be obtained by following the steps of the proof of Theorem 1. □

4.2. Elastic Net Penalty Problem

We consider the following linear regression model used in statistical learning:

$$y=A_1{\overline{u}}_1+A_2{\overline{u}}_2+\cdots +A_N{\overline{u}}_N+ϵ,$$

(47)

where y is the response predicted by the N predictors $A_1,A_2,\cdots ,A_N$ and $ϵ$ is a random error term. We assume that the model is sparse with a limited number of nonzero coefficients. The model can be written in matrix format as follows:

$$y=Au+ϵ,$$

(48)

where $y\in {\mathbb{R}}^M$, $A\in {\mathbb{R}}^{M\times N}$ is the predictor matrix and $u=({\overline{u}}_1,{\overline{u}}_2,\cdots ,{\overline{u}}_N)\in {\mathbb{R}}^N$. In the statistics community, one solves the model by recasting (48) as the following penalized optimization problem:

$$\underset{u\in {\mathbb{R}}^N}{min}{\parallel Au-y\parallel }_2^2+{\mathrm{pen}}_{\sigma }\left(u\right),$$

(49)

where ${\mathrm{pen}}_{\sigma }\left(u\right)$, $\sigma \ge 0$, is a penalty function of u.

Penalized regression and variable selection are two key topics in linear regression analysis. They have recently attracted the attention of many authors who have proposed and analyzed various penalty functions (see [39] and references therein). A very popular penalized regression model for variable selection is LASSO (least absolute shrinkage and selection operator), which was proposed by Tibshirani [40]. It is an ${\mathsf{\ell }}_1$-norm regularization least square model given by

$$\underset{u\in {\mathbb{R}}^N}{min}\frac{1}{2}{\parallel Au-y\parallel }_2^2+{\sigma }_1{\parallel u\parallel }_1,$$

(50)

where ${\sigma }_1\ge 0$ is a nonnegative regularization parameter. The ${\mathsf{\ell }}_1$ penalty makes LASSO perform both continuous shrinkage and automatic variable selection simultaneously [41]. However, its conditions are invalid when applied to a group of highly correlated variables [42]. In addition to LASSO, other approaches for variable selection with penalty functions more general than the ${\mathsf{\ell }}_1$ penalty have been proposed (please see [43] and references therein for more details on other penalty functions and applications to SVM).

In this subsection, we focus on the penalty function, which was proposed by Zou and Hastie [41] and called the elastic net penalty function. The elastic net is a linear combination of ${\mathsf{\ell }}_1$ and ${\mathsf{\ell }}_2$ penalties and is defined by

$${\mathrm{pen}}_{\sigma }\left(u\right):={\sigma }_1{\parallel u\parallel }_1+{\sigma }_2{\parallel u\parallel }_2^2,{\sigma }_1,{\sigma }_2>0.$$

Therefore, we will consider the following elastic net penalty problem:

$$\underset{u\in {\mathbb{R}}^N}{min}\frac{1}{2}{\parallel Au-y\parallel }_2^2+{\sigma }_1{\parallel u\parallel }_1+{\sigma }_2{\parallel u\parallel }_2^2.$$

(51)

Let

$$f\left(u\right)=\frac{1}{2}{\parallel Au-y\parallel }_2^2+{\sigma }_2{\parallel u\parallel }_2^2$$

and

$$g\left(u\right)={\sigma }_1{\parallel u\parallel }_1.$$

Then, $\nabla f\left(u\right)=A^{\top }(Au-y)+2{\sigma }_{2}u$, where $A^{\top }$ stands for the transpose of A. Thus, $\nabla f$ is monotone and Lipschitz continuous with constant $L+2{\sigma }_2$, L being the largest eigenvalue of the matrix $A^{\top }A$. Note that g is a proper lower semicontinuous convex function and $\partial g$ is maximal monotone. The resolvent of the maximal monotone operator $\partial g$ is given as (see [20])

$$J_{\sigma \partial g}v=arg\underset{u\in {\mathbb{R}}^N}{min}\{g\left(u\right)+\frac{1}{2\sigma }{\parallel u-v\parallel }_2^2\}.$$

By the optimality condition, the penalty problem (51) is equivalent to the following inclusion problem:

$$0\in \nabla f\left(u^{*}\right)+\partial g\left(u^{*}\right).$$

(52)

We apply our Algorithm 1 to solve the elastic net penalty problem (51), taking $S=\partial g$ and $T_i=T=\nabla f$ for each $i=1,2,\cdots ,N$, and then compare the effectiveness of our methods with some existing methods in the literature. To this end, let $u\in {\mathbb{R}}^N$ be a sparse randomly generated $N\times 1$ matrix and let $A\in {\mathbb{R}}^{M\times N}$ and $ϵ\in {\mathbb{R}}^M$ be randomly generated matrices, the entries of which are normally distributed with mean 0 and variance 1. In our experiments, we choose different values for $u,u_1$, A, and $ϵ$, as follows:

• Case A: $A=\mathrm{randn}(50,10)$, $ϵ=\mathrm{randn}(50,1)$, $u=\mathrm{sprandn}(10,1,0.3)$, $u_1=\mathrm{sprandn}(10,1,0.2)$.
• Case B: $A=\mathrm{randn}(1000,200)$, $ϵ=\mathrm{randn}(100,1)$, $u=\mathrm{sprandn}(200,1,0.1)$, $u_1=\mathrm{sprandn}(200,1,0.2)$.
• Case C: $A=\mathrm{randn}(200,1200)$, $ϵ=\mathrm{randn}(200,1)$, $u=\mathrm{sprandn}(1200,1,\frac{1}{60})$, $u_1=\mathrm{sprandn}(1200,1,0.02)$.
• Case D: $A=\mathrm{randn}(120,512)$, $ϵ=\mathrm{randn}(120,1)$, $u=\mathrm{sprandn}(512,1,\frac{1}{64})$, $u_1=\mathrm{sprandn}(512,1,\frac{1}{64})$.

We also choose ${\sigma }_1=0.6$, ${\sigma }_2=0.4$, $\mu =0.02$, ${\tau }_n=\frac{1}{\sqrt{n}+2}$, and $\mathcal{F}u=10u$ for all $u\in \mathcal{H}$. Using MATLAB 2021b, we compare the performance of our Algorithm 1 (RMFBSM) with MFBSM and Algorithm 2 in [14] (VTM).

Table 1. Numerical results.

		RMFBSM	MFBSM	VTM
Case A	CPU time (sec) No. of Iter.	0.0131 590	0.0200 936	0.0448 2818
Case B	CPU time (sec) No. of Iter.	0.5782 2441	0.6199 2734	2.6694 10,564
Case C	CPU time (sec) No. of Iter.	3.8084 10,710	5.3986 15,901	5.2824 15,345
Case D	CPU time (sec) No. of Iter.	0.8748 5412	1.1264 6833	0.9293 5530

and Figure 1 illustrate the outcome of our computations. The stopping criterion is $e_n=\frac{1}{n}{\parallel u_n-u\parallel }_2^2\le {10}^{-3}$.

5. Numerical Example

We present the following numerical example to further illustrate the effectiveness of our proposed Algorithm 1 (RMFBSM). The codes are written in MATLAB 2021b and run on an HP Laptop Windows 10 with Intel(R) Core(TM) i5 CPU and 4 GB RAM.

Example 1.

Consider ${\mathsf{\ell }}_2\left(\mathbb{R}\right):=\{u=(u_1,u_2,\cdots ,u_j,\cdots ),u_j\in \mathbb{R}:{\displaystyle \sum _{j=1}^{\infty }}|u_j{|}^2<\infty \}$, $\parallel u\parallel =({\displaystyle \sum _{j=1}^{\infty }}|u_j{{|}^2)}^{\frac{1}{2}}$$\forall u\in {\mathsf{\ell }}_2\left(\mathbb{R}\right)$. Let $\mathcal{H}={\mathsf{\ell }}_2\left(\mathbb{R}\right)$, and let $S:\mathcal{H}\rightrightarrows \mathcal{H}$, $T_i:\mathcal{H}\to \mathcal{H},i=1,2,3,$ and $\mathcal{F}:\mathcal{H}\to \mathcal{H}$ be defined by $Su=(2u_1,2u_2,\cdots ,2u_i,\cdots ),T_{1}x=(u_1,\frac{u_2}{2},\cdots ,\frac{u_j}{2},\cdots ),T_{2}u=\frac{u}{2}+(1,0,0,\cdots ,),T_{3}u=\frac{u}{3}+(2,0,0,\cdots ),\mathit{and}\mathcal{F}u=(7u_1,7u_2,\cdots ,7u_j,\cdots ),\mathit{respectively}\forall u\in \mathcal{H}.$ Choose $a_1=\frac{1}{2},a_2=\frac{1}{5}$, and $a_3=\frac{3}{10}$. One can see that S is maximal monotone, $T_i$s are Lipschitz continuous and monotone, and $\mathcal{F}$ is strongly monotone and Lipschitz continuous. In our computations, we choose, for each $n\in \mathbb{N}$, ${\tau }_n=\frac{1}{\sqrt{n+1}}$, ${\lambda }_1=0.3$, and $\mu =0.1$.

The following choices of the initial values $u_1$ are considered:

• Case a: $u_1=(-1,\frac{1}{2},-\frac{1}{4},\frac{1}{8},\cdots )$;
• Case b: $u_1=(\frac{2}{3},\frac{1}{9},\frac{1}{54},\frac{1}{324},\cdots )$;
• Case c: $u_1=(100,10,1,0.1,\cdots );$
• Case d: $u_1=(9,3\sqrt{3},3,\sqrt{3},\cdots ).$

We compare the performance of our Algorithm (1) (RMFBSM) with MFBSM and VTM. We choose ${\alpha }_n=\frac{1}{n+2}$ and $f\left(u\right)=\frac{u}{2}$ for VTM. The stopping criterion used for our computations is $e_n=\parallel u_n-J_{{\lambda }_{n}S}(I-{\lambda }_{n}T)u_n\parallel <{10}^{-8}$. Observe that $e_n=0$ implies that $u_n$ is a solution to the inclusion problem of finding $u\in \mathcal{H}$ such that $0\in (S+T)u$. We plot the graphs of errors against the number of iterations in each case. The figures and numerical results are shown in Figure 2 and

Table 2. Numerical results.

		RMFBSM	MFBSM	VTM
Case a	CPU time (sec) No of Iter.	0.0145 17	0.0155 70	0.0154 46
Case b	CPU time (sec) No. of Iter.	0.0040 17	0.0163 67	0.0050 45
Case c	CPU time (sec) No of Iter.	0.0019 23	0.0038 88	0.0048 58
Case d	CPU time (sec) No of Iter.	0.0019 19	0.0027 78	0.0359 52

, respectively.

Remark 4.

The numerical results show that our algorithm performs better than the two algorithms with which we compared it with respect to both the computation time taken and the number of iterations. Nevertheless, its performance can be further improved by incorporating inertial extrapolation terms into the algorithm. A possible future direction of research concerns estimating and comparing the rate of convergence of our proposed algorithm with the pertinent algorithms in the literature.

6. Conclusions

We study three classes of variational inclusion problems of finding a zero of the sum of monotone operators in real Hilbert space. We propose a unified simple structure, which combines a modification of the Tseng method with self-adaptive step sizes, and prove that the sequences it generates converge strongly to a solution of the problems. We apply our results to the elastic net penalty problem in statistical learning and to split feasibility problems. Numerical experiments are presented in order to illustrate the usefulness and effectiveness of our new method. Results related to accelerated versions of our method can be derived from the results of our recent work [27].