The Modified Viscosity Approximation Method with Inertial Technique and Forward–Backward Algorithm for Convex Optimization Model

Abstract

In this paper, we propose a new accelerated algorithm for finding a common fixed point of nonexpansive operators, and then, a strong convergence result of the proposed method is discussed and analyzed in real Hilbert spaces. As an application, we create a new accelerated viscosity forward–backward method (AVFBM) for solving nonsmooth optimization problems of the sum of two objective functions in real Hilbert spaces, and the strong convergence of AVFBM to a minimizer of the sum of two convex functions is established. We also present the application and simulated results of AVFBM for image restoration and data classification problems.

1. Introduction

Image restoration is a fundamental problem in image processing. The image restoration (image deblurring or image deconvolution) is concerned with the reconstruction or estimation of the uncorrupted image from a blurred and noisy image [1,2]. Thus, the main objective of the image restoration algorithms is to reduce the blurring effects and the noise that degraded the image by minimizing the noise of the degraded image to produce an estimate image that approaches the original image.

The image restoration problem can be modeled by a linear inverse problem, which is formulated by:

$$u=Bv+e,$$

(1)

where $B\in {\mathbb{R}}^{m\times n}$ is the blurring matrix, $v\in {\mathbb{R}}^n$ is an original image, $u\in {\mathbb{R}}^m$ is the observed image, and $e\in {\mathbb{R}}^m$ is a noise. One of the most popular models to solve Problem (1) is the least absolute shrinkage and selection operator (LASSO) [3], which can be considered in the following form:

$$\underset{v}{min}{\parallel Bv-u\parallel }_2^2+\tau {\parallel v\parallel }_1,$$

(2)

where $\tau >0$ is a regularization parameter, ${\parallel ·\parallel }_1$ is $l_1$-norm, and ${\parallel ·\parallel }_2$ is $l_2$-norm. Moreover, Problem (2) can be applied to solving many areas of science and applied science such as astronomical imaging [4], microscopy [5], and signal recovery problems [6].

The nonsmooth convex optimization model which includes (2) as a particular case has the following form:

$$\underset{x\in \mathcal{H}}{min}{\varphi }_1\left(x\right)+{\varphi }_2\left(x\right),$$

(3)

where $\mathcal{H}$ is a Hilbert space with norm $\parallel ·\parallel$, and inner product $\langle ·,·\rangle ,$${\varphi }_2:\mathcal{H}\to \mathbb{R}\cup \{\infty \}$ is a proper convex and lower semi-continuous function, and ${\varphi }_1:\mathcal{H}\to \mathbb{R}$ is convex differentiable with a Lipschitz continuous gradient constant $L>0.$ The solution set of Problem (3) will be denoted by $\mathrm{\Omega }:=Argmin({\varphi }_1+{\varphi }_2).$ Furthermore, x is a solution of Problem (3) if and only if x satisfies the fixed point equation:

$$x=pro{x}_{c{\varphi }_2}(I-c\nabla {\varphi }_1)\left(x\right),$$

(4)

where $c>0,$ I is an identity operator, $\nabla f$ is the gradient of $f,$ $pro{x}_{{\varphi }_2}={(I+\partial {\varphi }_2)}^{-1}$, and $\partial {\varphi }_2$ is the subdifferential of ${\varphi }_2$ defined by:

$$\partial {\varphi }_2\left(a^{*}\right):=\{u\in \mathcal{H}:{\varphi }_2\left(a\right)\ge \langle u,a-a^{*}\rangle +{\varphi }_2\left(a^{*}\right),a\in \mathcal{H}\},$$

see [7,8,9] for more details. For solving (3), the Forward–Backward splitting (FBS) algorithm [10] has been using the following form:

$$x_{k+1}=\underset{\underset{\mathrm{backward}\mathrm{step}}{︸}}{pro{x}_{c_k{\varphi }_2}}\underset{\mathrm{forward}\mathrm{step}}{\underbrace{(I-c_k\nabla {\varphi }_1)}}\left(x_k\right),k\in \mathbb{N},$$

(5)

where $x_1\in \mathcal{H}$ and $0<c_k<2/L.$ To accelerate the proximal gradient algorithm, the inertial technique or extrapolation technique was proposed by Nesterov in 1983 [11] for solving a class of convex optimization problems (3), where $F:={\varphi }_1+{\varphi }_2$ is a smooth and convex function. A typical algorithm takes the following form:

$$\left\{\begin{array}{c}{y}_k=x_k+{\theta }_k(x_k-x_{k-1}),\hfill \\ x_{k+1}=y_k+c\nabla F\left(y_k\right),k\in \mathbb{N},\hfill \end{array}\right.$$

(6)

where $c>0$ is the step size depending on the Lipschitz continuity modulus of $\nabla F$ and the inertial parameter ${\theta }_k\in (0,1)$ for all $k.$ He also showed that by choosing $\left\{{\theta }_k\right\}$ such that ${sup}_k{\theta }_k=1,$ this algorithm has a faster convergence rate than the general gradient algorithm; see [11]. In 2009, Beck and Teboulle [12] improved FBS by using the inertial techniques; this algorithm is known as the fast iterative shrinkage-thresholding algorithm (FISTA), which is defined as follows:

$$\left\{\begin{array}{c}{y}_k=pro{x}_{\frac{1}{L}{\varphi }_2}(x_k-\frac{1}{L}\nabla {\varphi }_1\left(x_k\right)),\hfill \\ t_{k+1}=\frac{1+\sqrt{1+4t_k^2}}{2},{\theta }_k=\frac{t_k-1}{t_{k+1}},\hfill \\ x_{k+1}=y_k+{\theta }_k(y_k-y_{k-1}),k\in \mathbb{N},\hfill \end{array}\right.$$

(7)

where $x_1=y_0\in \mathcal{H}, t_1=1$ and ${\theta }_k$ is the inertial parameter. The FISTA has been recognized as a fast method. It is noted that the inertial parameter $\left\{{\theta }_k\right\}$ in (7) satisfies ${sup}_k{\theta }_k=1.$ So, the sequence generated by FISTA has a rate of convergence that is proven to be significantly better, both theoretically and practically. Recently, Liang and Schonlieb [13] modified FISTA, called “FISTA-Mod”, for the short and proved weak convergence theorem of FISTA-Mod. Moreover, they proved $\parallel x_k-x_{k-1}\parallel =O(1/k).$ The FBS and FISTA are only weak convergence in Hilbert spaces. For strong convergence, the viscosity approximation method (VAM) of the fixed point of a nonexpansive operator T was proposed by Moudafi [14], who proved the strong convergence of the methods (8) in real Hilbert spaces.

$$x_{k+1}={\gamma }_{k}g\left(x_k\right)+(1-{\gamma }_k)T{x}_k,k\in \mathbb{N},$$

(8)

where $x_1\in \mathcal{H},$${\gamma }_k$ is a sequence in $(0,1)$ and g is a contraction operator. In 2008, Takahashi [15] modified the viscosity approximation method of Moudafi [14] for finding a common fixed point of a countable family of nonexpansive operators $\left\{T_k\right\}$. His algorithm takes the following form:

$$x_{k+1}={\gamma }_{k}g\left(x_k\right)+(1-{\gamma }_k)T_k{x}_k,k\in \mathbb{N},$$

(9)

where $x_1\in \mathcal{H},$$\left\{{\gamma }_k\right\}\subset (0,1)$, and g is a contraction operator. He proved a strong convergence theorem of (9) under some conditions on $\left\{T_k\right\}$ and $\left\{{\gamma }_k\right\}.$

In 2012, He and Guo [16] introduced the following modified viscosity approximation method for a countable family of nonexpansive operators:

$$x_{k+1}={\gamma }_{k}g\left(x_k\right)+(1-{\gamma }_k)L_k{x}_k,k\in \mathbb{N},$$

(10)

where $\left\{{\gamma }_k\right\}\subset (0,1),$ $L_k={\sum }_{i=1}^k\left(\frac{w_i}{s_k}\right)T_i,$ $s_k={\sum }_{i=1}^k{w}_i,w_i>0$ with ${\sum }_{i=k}^{\infty }{w}_k=1.$ They proved strongly the convergence of (10) under the condition on $\left\{{\gamma }_k\right\}$ without any other condition on $\left\{T_k\right\}.$ However, this algorithm needs larger computational work than that of (9). After that, several algorithms for the common fixed points of a countable family of nonexpansive operators were introduced and discussed; see [16,17,18,19,20].

Inspired by [10,12,15], in this paper, we propose a simple method with the inertial technique for solving a common fixed point problem of a countable family of nonexpansive operators in a real Hilbert space. We then prove a strong convergence of the proposed method under some suitable conditions. Finally, we apply our proposed method to solving the image restoration and classification problems.

The rest of this paper is organized as follows: In Section 2, we present some notation and useful lemmas that will be used in this paper. The strong convergence of the accelerated viscosity fixed point method and the accelerated viscosity forward–backward method are analyzed in Section 3. Applications and simulated results for image restoration and data classification problems are given in Section 4. Finally, we give a conclusion remark for further study in Section 5.

2. Preliminaries

In this section, we present some definitions and useful lemmas for proving our main results in the next section. Throughout this paper, we adopt the following notations:

• $\mathcal{H}$ denotes a real Hilbert space with norm $\parallel ·\parallel$ and inner product $\langle ·,·\rangle ;$
• C denotes a nonempty closed convex subset of $\mathcal{H};$
• $Fix\left(T\right)$ denotes the set of all fixed points of $T;$
• ⇀ and → denote the weak convergence and strong convergence, respectively;
• $pro{x}_{c{\varphi }_2}(I-c\nabla {\varphi }_1)$ denotes the forward–backward operator of ${\varphi }_1$ and ${\varphi }_2$ with respect to $c.$

A mapping $T:C\to C$ is said to be an L-Lipschitz operator if there exists $L>0$ such that $\parallel Ta-Tb\parallel \le L\parallel a-b\parallel$ for all $a,b\in C.$ An L-Lipschitz operator is called a nonexpansive operator and contraction operator if $L=1$ and $L\in (0,1),$ respectively. If $T:C\to C$ is a nonexpansive operator with $Fix\left(T\right)\ne \varnothing ,$ then $Fix\left(T\right)$ is closed and convex, and the mapping $I-T$ is demiclosed at zero, that is for any sequence $\left\{x_k\right\}\subset C$ such that $x_k\rightharpoonup a$ and $\parallel x_k-T{x}_k\parallel \to 0$ imply $a\in Fix\left(T\right).$ A mapping $P_C$ is said to be a metric projection of $\mathcal{H}$ onto $C,$ if for every $a\in \mathcal{H},$ there exists a unique nearest point in C denoted by $P_{C}a$ such that:

$$\parallel a-P_{C}a\parallel \le \parallel a-b\parallel ,\forall b\in C.$$

Moreover, $P_C$ is firmly nonexpansive mapping and $P_C$ satisfying $\langle a-P_{C}a,b-P_{C}a\rangle \le 0,\forall a\in \mathcal{H},b\in C.$ Let $\left\{T_k\right\}$ and $\mathrm{\Lambda }$ be families of nonexpansive operators of C into itself such that $\varnothing \ne Fix\left(\mathrm{\Lambda }\right)\subset \mathrm{\Gamma }:={\cap }_{k=1}^{\infty }Fix\left(T_k\right),$ where $Fix\left(\mathrm{\Lambda }\right)$ is the set of all common fixed points of $\mathrm{\Lambda }.$ A sequence $\left\{T_k\right\}$ is said to satisfy the NST-condition (I) with $\mathrm{\Lambda }$ [21] if for every bounded sequence $\left\{x_k\right\}$ in $C,$

$$\underset{k\to \infty }{lim}\parallel x_k-T_k{x}_k\parallel =0\mathrm{implies}\underset{k\to \infty }{lim}\parallel x_k-T{x}_k\parallel =0\mathrm{for}\mathrm{all}T\in \mathrm{\Lambda }.$$

If $\mathrm{\Lambda }$ is singleton, i.e., $\mathrm{\Lambda }=\left\{T\right\},$ then $\left\{T_k\right\}$ is said to satisfy the NST-condition (I) with $T.$ After that, Aoyama, Kohsaka and Takahashi [22] introduced the condition (Z), which is more general than that of NST-condition (I). A sequence $\left\{T_k\right\}$ is said to satisfy the condition (Z) if whenever $\left\{x_k\right\}$ is a bounded sequence in C such that ${lim}_{k\to \infty }\parallel x_k-T_k{x}_k\parallel =0$, it follows that every weak cluster point of $\left\{x_k\right\}$ belongs to $\mathrm{\Gamma }.$

It is also known that $pro{x}_{c{\varphi }_2}(I-c\nabla {\varphi }_1)$ is a nonexpansive mapping when $0<c<2/L.$ The following lemmas are useful for proving our main results.

Lemma 1

([23]). Let ${\varphi }_1:\mathcal{H}\to \mathbb{R}$ be a convex and differentiable function with an L-Lipschitz continuous gradient of ${\varphi }_1$ and let ${\varphi }_2:\mathcal{H}\to \mathbb{R}\cup \{\infty \}$ be a proper lower semi-continuous and convex function. Let $T_k:=pro{x}_{c_k{\varphi }_2}(I-c_k\nabla {\varphi }_1)$ and $T:=pro{x}_{c{\varphi }_2}(I-c\nabla {\varphi }_1)$, where $c_k,c\in (0,2/L)$ with $c_k\to c$ as $k\to \infty$. Then, $\left\{T_k\right\}$ satisfies NST-condition (I) with T.

Lemma 2

([24]). For all $a,b\in \mathcal{H},$ and $t\in [0,1]$ the following hold:

(i)

${\parallel ta+(1-t)b\parallel }^2={t\parallel a\parallel }^2+{(1-t)\parallel b\parallel }^2-t(1-t){\parallel a-b\parallel }^2;$

(ii)

${\parallel a\pm b\parallel }^2={\parallel a\parallel }^2\pm 2\langle a,b\rangle +{\parallel b\parallel }^2;$

(iii)

${\parallel a+b\parallel }^2={\parallel a\parallel }^2+2\langle b,a+b\rangle .$

Lemma 3

([25]). Let $\{a_i,i=1,2,\dots ,k\}\subset \mathcal{H}.$ For $b_i\in (0,1),i=1,2,\dots ,k$ such that ${\sum }_{i=1}^k{b}_i=1.$ Then, the following identity holds:

$${∥\sum _{i=1}^k{b}_i{a}_i∥}^2=\sum _{i=1}^k{b}_i\parallel a_i{\parallel }^2-\sum _{i,j=1,i\ne j}^k{b}_i{b}_j{\parallel a_i-a_j\parallel }^2.$$

Lemma 4

([26]). Let $\left\{a_k\right\}$ be a sequence of non-negative real numbers, $\left\{b_k\right\}$ be a sequence of real numbers, and $\left\{t_k\right\}$ be a sequence of real numbers in $(0,1)$ such that ${\sum }_{n=1}^{\infty }{t}_k=\infty .$ Assume that:

$$a_{k+1}\le (1-t_k)a_k+t_k{b}_k,k\in \mathbb{N}.$$

If ${lim\; sup}_{i\to \infty }{b}_{k_i}\le 0$ for every subsequence $\left\{a_{k_i}\right\}$ of $\left\{a_k\right\}$ satisfying the condition:

$$\underset{i\to \infty }{lim\; inf}(a_{k_i+1}-a_{k_i})\ge 0,$$

then ${lim}_{k\to \infty }{a}_k=0.$

3. Main Results

In this section, we propose a new accelerated viscosity fixed point method, which is called “AVFPM” for solving a common fixed point of nonexpansive operators in a real Hilbert space. In order to introduce AVFPM, we assume the following:

• $g:\mathcal{H}\to \mathcal{H}$ is a contraction with constant $\eta \in (0,1);$
• $\{T_k:\mathcal{H}\to \mathcal{H}\}$ is a family of nonexpansive operators;
• $\left\{T_k\right\}$ satisfies condition (Z);
• $\mathrm{\Gamma }:={\cap }_{k=1}^{\infty }Fix\left(T_k\right)\ne \varnothing .$

Theorem 5.

Let $\left\{x_k\right\}$ be a sequence generated by Algorithm 1 (AVFPM). Then, $\left\{x_k\right\}$ converges strongly to an element $a^{*}\in \mathrm{\Gamma },$ where $a^{*}=P_{\mathrm{\Gamma }}g\left(a^{*}\right).$

Now, we prove the strong convergence of Algorithm 1 (AVFPM).

Algorithm 1: An accelerated viscosity fixed point method (AVFPM).

Initialization: Take $x_0,x_1\in \mathcal{H}$ arbitrarily and positive sequences $\left\{{\lambda }_k\right\},\left\{{\sigma }_k\right\},$$\left\{{\gamma }_k\right\},\left\{{\beta }_k\right\}$, and $\left\{{\alpha }_k\right\}$ satisfy the following conditions: $$\begin{array}{cc}\hfill \left\{{\alpha }_k\right\}\subset (0,1),& \underset{k\to \infty }{lim}{\alpha }_k=0\mathrm{and}\sum _{k=1}^{\infty }{\alpha }_k=\infty ,\hfill \\ \hfill \left\{{\beta }_k\right\}\subset (0,1),0& <a_5\le {\gamma }_k<1,{\alpha }_k+{\beta }_k+{\gamma }_k=1,\hfill \\ \hfill 0<a_1\le {\lambda }_k\le & a_2<1,0<a_3\le {\sigma }_k\le a_4<1.\hfill \end{array}$$ for some positive real numbers $a_1,a_2,a_3$, and $a_4.$ Iterative steps: Calculate $x_{k+1}$ as follows: Step 1. Choose a bounded sequence of non-negative real numbers $\left\{{\mu }_k\right\}.$ For $k\ge 1,$ set $${\theta }_k=\left\{\begin{array}{cc}min\left\{{\mu }_k,{\displaystyle \frac{{\tau }_k}{\parallel x_k-x_{k-1}\parallel }}\right\}\hfill & \mathrm{if}{x}_k\ne x_{k-1},\hfill \\ {\mu }_k\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$ where $\left\{{\tau }_k\right\}$ is a sequence of positive real numbers such that ${lim}_{k\to \infty }{\tau }_k/{\alpha }_k=0.$ Step 2. Compute $$\left\{\begin{array}{c}{w}_k=x_k+{\theta }_k(x_k-x_{k-1}),\hfill \\ z_k=(1-{\lambda }_k)w_k+{\lambda }_k{T}_k{w}_k,\hfill \\ y_k=(1-{\sigma }_k)T_k{w}_k+{\sigma }_k{T}_k{z}_k,\hfill \\ x_{k+1}={\alpha }_{k}g\left(w_k\right)+{\beta }_k{T}_k{z}_k+{\gamma }_k{T}_k{y}_k.\hfill \end{array}\right.$$ Update $k:=k+1$ and return to Step 1.

Proof. 

By the Banach contraction principle, there exists a unique $a^{*}\in \mathrm{\Gamma }$ such that $a^{*}=P_{\mathrm{\Gamma }}g\left(a^{*}\right).$ By definitions of $x_{k+1},$ we have:

$$\begin{array}{cc}\hfill \parallel w_k-a^{*}\parallel & \le \parallel x_k-a^{*}\parallel +{\theta }_k\parallel x_k-x_{k-1}\parallel ,\hfill \end{array}$$

(11)

and:

$$\begin{array}{cc}\hfill \parallel z_k-a^{*}\parallel & \le (1-{\lambda }_k)\parallel w_k-a^{*}\parallel +{\lambda }_k\parallel T_k{w}_k-a^{*}\parallel \le \parallel w_k-a^{*}\parallel .\hfill \end{array}$$

(12)

From (12), we get:

$$\begin{array}{cc}\hfill \parallel y_k-a^{*}\parallel & \le (1-{\sigma }_k)\parallel T_k{w}_k-a^{*}\parallel +{\sigma }_k\parallel T_k{z}_k-a^{*}\parallel \le \parallel w_k-a^{*}\parallel .\hfill \end{array}$$

(13)

From (11)–(13), we obtain:

$$\begin{array}{cc}\hfill \parallel x_{k+1}-a^{*}\parallel & \le {\alpha }_k\parallel g\left(w_k\right)-g\left(a^{*}\right)\parallel +{\alpha }_k\parallel g\left(a^{*}\right)-a^{*}\parallel \hfill \\ & +{\beta }_k\parallel T_k{z}_k-a^{*}\parallel +{\gamma }_k\parallel T_k{y}_k-a^{*}\parallel \hfill \\ \hfill & \le {\alpha }_k\eta \parallel w_k-a^{*}\parallel +{\alpha }_k\parallel g\left(a^{*}\right)-a^{*}\parallel \hfill \\ & +{\beta }_k\parallel z_k-a^{*}\parallel +{\gamma }_k\parallel y_k-a^{*}\parallel \hfill \\ \hfill & \le (1-{\alpha }_k(1-\eta ))\parallel w_k-a^{*}\parallel +{\alpha }_k\parallel g\left(a^{*}\right)-a^{*}\parallel \hfill \\ \hfill & \le (1-{\alpha }_k(1-\eta ))\parallel x_k-a^{*}\parallel \hfill \\ \hfill & +{\alpha }_k\left[{\displaystyle \frac{{\theta }_k}{{\alpha }_k}}\parallel x_k-x_{k-1}\parallel +\parallel g\left(a^{*}\right)-a^{*}\parallel \right].\hfill \end{array}$$

By the condition of ${\theta }_k$, we have ${lim}_{k\to \infty }{\displaystyle \frac{{\theta }_k}{{\alpha }_k}}\parallel x_k-x_{k-1}\parallel =0,$ and so there exists a constant $M>0$ such that ${\displaystyle \frac{{\theta }_k}{{\alpha }_k}}\parallel x_k-x_{k-1}\parallel \le M$$\forall k\ge 1.$ Thus:

$$\begin{array}{c}\hfill \parallel x_{k+1}-a^{*}\parallel \le (1-{\alpha }_k(1-\eta ))\parallel x_k-a^{*}\parallel +{\alpha }_k(M+\parallel g\left(a^{*}\right)-a^{*}\parallel ).\end{array}$$

By mathematical induction, we get:

$$\begin{array}{c}\hfill \parallel x_{k+1}-a^{*}\parallel \le max\left\{\parallel x_1-a^{*}\parallel ,{\displaystyle \frac{M+\parallel g\left(a^{*}\right)-a^{*}\parallel }{1-\eta }}\right\}\forall k\ge 1.\end{array}$$

This implies that $\left\{x_k\right\}$ is bounded and $\left\{w_k\right\},\left\{z_k\right\},\left\{y_k\right\},\left\{T_k{w}_k\right\},\left\{T_k{z}_k\right\},$

$\left\{T_k{y}_k\right\},and\left\{g\left(w_k\right)\right\}$ are also bounded. By Lemma 2, we obtain:

$$\begin{array}{cc}\hfill \parallel w_k-a^{*}{\parallel }^2& \le \parallel x_k-a^{*}{\parallel }^2+{\theta }_k^2{\parallel x_k-x_{k-1}\parallel }^2\hfill \\ \hfill & +2{\theta }_k\parallel x_k-a^{*}\parallel \parallel x_k-x_{k-1}\parallel ,\hfill \end{array}$$

(14)

and:

$$\begin{array}{cc}\hfill \parallel z_k-a^{*}{\parallel }^2& \le \parallel w_k-a^{*}{\parallel }^2-{\lambda }_k(1-{\lambda }_k){\parallel w_k-T_k{w}_k\parallel }^2.\hfill \end{array}$$

(15)

By Lemma 2(i) and (15), we obtain:

$$\begin{array}{cc}\hfill \parallel y_k-a^{*}{\parallel }^2& \le (1-{\sigma }_k)\parallel T_k{w}_k-a^{*}{\parallel }^2+{\sigma }_k{\parallel T_k{z}_k-a^{*}\parallel }^2\hfill \\ & -{\sigma }_k(1-{\sigma }_k){\parallel T_k{w}_k-T_k{z}_k\parallel }^2\hfill \\ \hfill & \le \parallel w_k-a^{*}{\parallel }^2-{\sigma }_k{\lambda }_k(1-{\lambda }_k){\parallel w_k-T_k{w}_k\parallel }^2\hfill \\ & -{\sigma }_k(1-{\sigma }_k){\parallel T_k{w}_k-T_k{z}_k\parallel }^2\hfill \end{array}$$

(16)

From (12), (14), (16), Lemmas 2(iii) and 3, we have:

$$\begin{array}{cc}\hfill \parallel x_{k+1}-a^{*}{\parallel }^2& =\parallel {\alpha }_k(g\left(w_k\right)-g\left(a^{*}\right))+{\beta }_k(T_k{z}_k-a^{*})+{\gamma }_k(T_k{y}_k-a^{*}){\parallel }^2\hfill \\ & +2{\alpha }_k\langle g\left(a^{*}\right)-a^{*},x_{k+1}-a^{*}\rangle \hfill \\ \hfill & \le {\alpha }_k\parallel g\left(w_k\right)-g\left(a^{*}\right){\parallel }^2+{\beta }_k\parallel T_k{z}_k-a^{*}{\parallel }^2+{\gamma }_k{\parallel T_k{y}_k-a^{*}\parallel }^2\hfill \\ & +2{\alpha }_k\langle g\left(a^{*}\right)-a^{*},x_{k+1}-a^{*}\rangle \hfill \\ \hfill & \le {\alpha }_k\eta \parallel w_k-a^{*}{\parallel }^2+{\beta }_k\parallel w_k-a^{*}{\parallel }^2+{\gamma }_k{\parallel w_k-a^{*}\parallel }^2\hfill \\ \hfill & -{\gamma }_k{\sigma }_k{\lambda }_k(1-{\lambda }_k){\parallel w_k-T_k{w}_k\parallel }^2\hfill \\ & -{\gamma }_k{\sigma }_k(1-{\sigma }_k){\parallel T_k{w}_k-T_k{z}_k\parallel }^2+2{\alpha }_k\langle g\left(a^{*}\right)-a^{*},x_{k+1}-a^{*}\rangle \hfill \\ \hfill & \le (1-{\alpha }_k(1-\eta )){\parallel x_k-a^{*}\parallel }^2\hfill \\ & +{\alpha }_k[2\parallel x_k-a^{*}\parallel \left(\frac{{\theta }_k}{{\alpha }_k}\parallel x_k-x_{k-1}\parallel \right)\hfill \\ & +\left(\frac{{\theta }_k}{{\alpha }_k}\parallel x_k-x_{k-1}\parallel \right){\theta }_k\parallel x_k-x_{k-1}\parallel \hfill \\ \hfill & +2\langle g\left(a^{*}\right)-a^{*},x_{k+1}-a^{*}\rangle ]-{\gamma }_k{\sigma }_k{\lambda }_k(1-{\lambda }_k){\parallel w_k-T_k{w}_k\parallel }^2\hfill \\ & -{\gamma }_k{\sigma }_k(1-{\sigma }_k){\parallel T_k{w}_k-T_k{z}_k\parallel }^2.\hfill \end{array}$$

(17)

So, we get:

$$\begin{array}{c}\hfill {\gamma }_k{\sigma }_k{\lambda }_k(1-{\lambda }_k)\parallel w_k-T_k{w}_k{\parallel }^2\le \parallel x_k-a^{*}{\parallel }^2-{\parallel x_{k+1}-a^{*}\parallel }^2+{\alpha }_k{M}^{{}^{\prime }},\end{array}$$

(18)

and:

$$\begin{array}{c}\hfill {\gamma }_k{\sigma }_k(1-{\sigma }_k)\parallel T_k{w}_k-T_k{z}_k{\parallel }^2\le \parallel x_k-a^{*}{\parallel }^2-{\parallel x_{k+1}-a^{*}\parallel }^2+{\alpha }_k{M}^{{}^{\prime }},\end{array}$$

(19)

where:

$M^{{}^{\prime }}={sup}_{n\ge 1}\left\{2\parallel x_k-a^{*}\parallel \left(\frac{{\theta }_k}{{\alpha }_k}\parallel x_k-x_{k-1}\parallel \right)+\left(\frac{{\theta }_k}{{\alpha }_k}\parallel x_k-x_{k-1}\parallel \right){\theta }_k\parallel x_k-x_{k-1}\parallel +2\langle g\left(a^{*}\right)-a^{*},x_{k+1}-a^{*}\rangle \right\}.$

Now, we show that $\left\{x_k\right\}$ converges strongly to $a^{*}.$ Let $a_k={\parallel x_k-a^{*}\parallel }^2.$ Suppose that $\left\{a_{k_i}\right\}$ is a subsequence of $\left\{a_k\right\}$ such that ${lim\; inf}_{i\to \infty }(a_{k_i+1}-a_{k_i})\ge 0.$ By (19) and conditions of $\left\{{\lambda }_k\right\},\left\{{\sigma }_k\right\},$$\left\{{\gamma }_k\right\},\left\{{\beta }_k\right\}$, and $\left\{{\alpha }_k\right\},$ we have:

$$\begin{array}{cc}\hfill \underset{i\to \infty }{lim\; sup}{\gamma }_{k_i}{\sigma }_{k_i}(1-{\sigma }_{k_i}){\parallel T_{k_i}{w}_{k_i}-T_{k_i}{z}_{k_i}\parallel }^2& \le \underset{i\to \infty }{lim\; sup}(a_{k_i}-a_{k_i+1}+{\alpha }_{k_i}{M}^{{}^{\prime }})\hfill \\ \hfill & \le \underset{i\to \infty }{lim\; sup}(a_{k_i}-a_{k_i+1})\hfill \\ & +\underset{i\to \infty }{lim\; sup}{\alpha }_{k_i}{M}^{{}^{\prime }}\hfill \\ \hfill & =-\underset{i\to \infty }{lim\; inf}(a_{k_i+1}-a_{k_i})\hfill \\ & \le 0.\hfill \end{array}$$

(20)

This implies that:

$$\underset{i\to \infty }{lim}\parallel T_{k_i}{w}_{k_i}-T_{k_i}{z}_{k_i}\parallel =0.$$

(21)

Similarly, we have ${lim}_{i\to \infty }\parallel w_{k_i}-T_{k_i}{w}_{k_i}\parallel =0.$ By definitions of ${\theta }_k$ and $x_{k+1},$ we get:

$$\begin{array}{c}\hfill \underset{i\to \infty }{lim}\parallel w_{k_i}-x_{k_i}\parallel =0.\end{array}$$

(22)

So, we obtain:

$$\begin{array}{cc}\hfill \parallel x_{k_i}-T_{k_i}{x}_{k_i}\parallel & \le \parallel x_{k_i}-T_{k_i}{w}_{k_i}\parallel +\parallel T_{k_i}{w}_{k_i}-T_{k_i}{x}_{k_i}\parallel \hfill \\ \hfill & \le 2\parallel w_{k_i}-x_{k_i}\parallel +\parallel w_{k_i}-T_{k_i}{w}_{k_i}\parallel \to 0.\hfill \end{array}$$

(23)

From definitions of $x_{k+1},$ we have:

$$\begin{array}{cc}\hfill \parallel z_{k_i}-x_{k_i}\parallel & \le \parallel w_{k_i}-x_{k_i}\parallel +{\lambda }_{k_i}\parallel w_{k_i}-T_{k_i}{w}_{k_i}\parallel ,\hfill \\ \hfill \parallel y_{k_i}-T_{k_i}{w}_{k_i}\parallel & ={\sigma }_{k_i}\parallel T_{k_i}{w}_{k_i}-T_{k_i}{z}_{k_i}\parallel ,\hfill \end{array}$$

(24)

and:

$$\begin{array}{cc}\hfill \parallel y_{k_i}-x_{k_i}\parallel & \le \parallel y_{k_i}-T_{k_i}{w}_{k_i}\parallel +\parallel T_{k_i}{w}_{k_i}-w_{k_i}\parallel +\parallel w_{k_i}-x_{k_i}\parallel .\hfill \end{array}$$

(25)

This implies:

$$\begin{array}{c}\hfill \underset{i\to \infty }{lim}\parallel z_{k_i}-x_{k_i}\parallel =\underset{i\to \infty }{lim}\parallel y_{k_i}-x_{k_i}\parallel =0.\end{array}$$

(26)

Moreover,

$$\begin{array}{cc}\hfill \parallel x_{k_i+1}-x_{k_i}\parallel & \le \parallel x_{k_i+1}-T_{k_i}{x}_{k_i}\parallel +\parallel T_{k_i}{x}_{k_i}-x_{k_i}\parallel \hfill \\ \hfill & \le {\alpha }_{k_i}\parallel g\left(w_{k_i}\right)-T_{k_i}{x}_{k_i}\parallel +{\beta }_{k_i}\parallel z_{k_i}-x_{k_i}\parallel \hfill \\ & +{\gamma }_{k_i}\parallel y_{k_i}-x_{k_i}\parallel +\parallel T_{k_i}{x}_{k_i}-x_{k_i}\parallel ,\hfill \end{array}$$

(27)

which implies ${lim}_{i\to \infty }\parallel x_{k_i+1}-x_{k_i}\parallel =0.$ Now, we claim:

$$\underset{i\to \infty }{lim\; sup}\langle g\left(a^{*}\right)-a^{*},x_{k_i+1}-a^{*}\rangle \le 0.$$

Indeed, choose a subsequence $\left\{x_{k_{i_j}}\right\}$ of $\left\{x_{k_i}\right\}$ such that:

$$\underset{i\to \infty }{lim\; sup}\langle g\left(a^{*}\right)-a^{*},x_{k_i}-a^{*}\rangle =\underset{j\to \infty }{lim}\langle g\left(a^{*}\right)-a^{*},x_{k_{i_j}}-a^{*}\rangle .$$

Since $\left\{x_{k_{i_j}}\right\}$ is bounded, there exists a subsequence $\left\{x_{k_{i_{j_p}}}\right\}$ of $\left\{x_{k_{i_j}}\right\}$ such that $x_{k_{i_{j_p}}}\rightharpoonup u\in H.$ Without loss of generality, we may assume that $x_{k_{i_j}}\rightharpoonup u\in \mathcal{H}.$ Since $\left\{T_k\right\}$ satisfies condition (Z), we have $u\in \mathrm{\Gamma }.$ As ${lim}_{i\to \infty }\parallel x_{k_i+1}-x_{k_i}\parallel =0$ and $a^{*}=P_{\mathrm{\Gamma }}g\left(a^{*}\right),$ we obtain:

$$\underset{i\to \infty }{lim\; sup}\langle g\left(a^{*}\right)-a^{*},x_{k_i+1}-a^{*}\rangle =\langle g\left(a^{*}\right)-a^{*},u-a^{*}\rangle \le 0.$$

(28)

By (17), (28), and ${lim}_{k\to \infty }{\displaystyle \frac{{\theta }_k}{{\alpha }_k}}\parallel x_k-x_{k-1}\parallel =0,$ we can apply Lemma 4 to obtain ${lim}_{k\to \infty }\parallel x_k-a^{*}\parallel =0;$ that is, $\left\{x_k\right\}$ converges strongly to $a^{*}=P_{\mathrm{\Gamma }}g\left(a^{*}\right).$ This completes the proof.    □

Finally, we will apply the Algorithm 1 (AVFPM) for solving the nonsmooth convex optimization problems (3) of the sum of two objective functions ${\varphi }_1$ and ${\varphi }_2$ by assuming the following:

• $g:\mathcal{H}\to \mathcal{H}$ is a contraction with constant $\eta \in (0,1);$
• ${\varphi }_1:\mathcal{H}\to \mathbb{R}$ is convex differentiable with Lipschitz continuous gradient constant $L>0;$
• ${\varphi }_2:\mathcal{H}\to \mathbb{R}\cup \{\infty \}$ is a proper convex and lower semi-continuous function;
• $\mathrm{\Omega }:=Argmin({\varphi }_1+{\varphi }_2)\ne \varnothing .$

By setting $T_k=pro{x}_{c_k{\varphi }_2}(I-c_k\nabla {\varphi }_1),$ which is the forward–backward operator of ${\varphi }_1$ and ${\varphi }_2$ with respect to $c_k\in (0,2/L)$ and $c_k\to c,$ we have an accelerated viscosity forward–backward method for solving the problems (3) as follows:

Next, we prove the strong convergence of Algorithm 2 (AVFBM) by using Theorem 5.

Algorithm 2: An accelerated viscosity forward–backward method (AVFBM).

Initialization: Take $x_0,x_1\in \mathcal{H}$ arbitrarily and positive sequences $\left\{{\lambda }_k\right\},\left\{{\sigma }_k\right\},\left\{{\gamma }_k\right\},\left\{{\beta }_k\right\},$ and $\left\{{\alpha }_k\right\}$ satisfy the following conditions: $$\begin{array}{cc}\hfill \left\{{\alpha }_k\right\}\subset (0,1),& \underset{k\to \infty }{lim}{\alpha }_k=0\mathrm{and}\sum _{k=1}^{\infty }{\alpha }_k=\infty ,\hfill \\ \hfill \left\{{\beta }_k\right\}\subset (0,1),0& <a_5\le {\gamma }_k<1,{\alpha }_k+{\beta }_k+{\gamma }_k=1,\hfill \\ \hfill 0<a_1\le {\lambda }_k\le & a_2<1,0<a_3\le {\sigma }_k\le a_4<1.\hfill \end{array}$$ for some positive real numbers $a_1,a_2,a_3,$ and $a_4.$ Iterative steps: Calculate $x_{k+1}$ as follows: Step 1. Choose a bounded sequence of non-negative real numbers $\left\{{\mu }_k\right\}.$ For $k\ge 1,$ defined ${\theta }_k$ by the same as Algorithm 1. Step 2. Compute $$\left\{\begin{array}{c}{w}_k=x_k+{\theta }_k(x_k-x_{k-1}),\hfill \\ z_k=(1-{\lambda }_k)w_k+{\lambda }_{k}pro{x}_{c_k{\varphi }_2}(I-c_k\nabla {\varphi }_1)w_k,\hfill \\ y_k=(1-{\sigma }_k)pro{x}_{c_k{\varphi }_2}(I-c_k\nabla {\varphi }_1)w_k+{\sigma }_{k}pro{x}_{c_k{\varphi }_2}(I-c_k\nabla {\varphi }_1)z_k,\hfill \\ x_{k+1}={\alpha }_{k}g\left(w_k\right)+{\beta }_{k}pro{x}_{c_k{\varphi }_2}(I-c_k\nabla {\varphi }_1)z_k\hfill \\ +{\gamma }_{k}pro{x}_{c_k{\varphi }_2}(I-c_k\nabla {\varphi }_1)y_k.\hfill \end{array}\right.$$ Update $k:=k+1$ and return to Step 1.

Theorem 6.

Let $\left\{x_k\right\}$ be a sequence generated by Algorithm 2 (AVFBM). Then, $\left\{x_k\right\}$ converges strongly to an element $a^{*}\in \mathrm{\Omega },$ where $a^{*}=P_{\mathrm{\Omega }}g\left(a^{*}\right).$

Proof. 

Let $T:=pro{x}_{c{\varphi }_2}(I-c\nabla {\varphi }_1)$ and $T_k:=pro{x}_{c_k{\varphi }_2}(I-c_k\nabla {\varphi }_1).$ Then, T and $\left\{T_k\right\}$ are nonexpansive operators for all $k,$ and $Fix\left(T\right)={\cap }_{k=1}^{\infty }Fix\left(T_k\right)=Argmin({\varphi }_1+{\varphi }_2).$ By Lemma 1, we have $\left\{T_k\right\},$ which satisfies condition (Z). Therefore, we obtain the result directly by Theorem 5. □

4. Application and Simulated Results

4.1. Image Restoration

In this example, we apply Algorithm 2 (AVFBM) to solving an image restoration problem (2) and compare the deblurring efficiency of AVFBM, FBS [10], and FISTA [12]. Our programs are written in MATLAB and run on a laptop with an Intel core i5, 4.00 GB RAM, and windows 8 (64-bit). All algorithms applied to the $l_1$-regularization problem (2); that is, ${\varphi }_1\left(x\right)={\parallel Bx-b\parallel }_2^2$ and ${\varphi }_2\left(x\right)=\tau {\parallel x\parallel }_1,$ where B is the blurring operator, b is the observed image, and $\tau$ is the regularization parameter. The maximum iteration number for all methods was fixed at 500.

In these experiments, we consider four gray-scale images (Cameraman, Lenna, Woman, and Boy) with size of $256\times 256$ as the original images and consider Gaussian blur of filter size $9\times 9$ with a standard deviation $\sigma =4$ with noise ${10}^{-4}.$ We have measured the performance of AVFBM, FBS, and FISTA by means of the Signal-to-Noise Ratio (SNR) [27] and Peak Signal-to-Noise Ratio (PSNR) [28]. The SNR and PSNR at $x_k$ of the restored images are defined as:

$$SNR(x,x_k)=10{log}_{10}\left\{\frac{\parallel x-\overline{x}{\parallel }_2}{\parallel x_k{-x\parallel }_2}\right\},$$

$$PSNR\left(x_k\right)=10{log}_{10}\left(\frac{{255}^2}{MSE}\right),$$

where $MSE=\frac{1}{{256}^2}{\parallel x_k-x\parallel }_2^2,$x is the original image, and $\overline{x}$ is the mean of the original image. The regularization parameter was chosen to be $\tau ={10}^{-4},$ and the initial image was the blurred image. The Lipschitz constant L of the gradient $\nabla f$ is $L=2{\lambda }_{max}\left(B^{T}B\right)$ [12]. The parameters of the algorithms are chosen as follows: ${\lambda }_k=\frac{0.5k}{k+1},{\sigma }_k=\frac{0.99k}{k+1},{\alpha }_k=\frac{1}{50k},{\beta }_k=\frac{1}{300k+1},{\gamma }_k=1-{\alpha }_k-{\beta }_k,$$c_k=\frac{k}{L(k+1)},c=\frac{1}{L},{\tau }_k=\frac{{10}^{15}}{k^2}\mathrm{and}{\mu }_k=\frac{t_k-1}{t_{k+1}},$ where $t_k$ is a sequence defined by $t_1=1$ and $t_{k+1}=\frac{1+\sqrt{1+4t_k^2}}{2}.$ The contraction mapping is defined by $g\left(a\right)=0.95a$ for all $a\in {\mathbb{R}}^n.$ The comparison of the performance of AVFBM, FISTA, and FBS by means of SNR and PSNR is shown in Figure 1. The plot of SNR and PSNR at $x_k$ of the restored images is shown in Figure 2. We see from Figure 1 and Figure 2 that AVFBM gives a higher performance of SNR and PSNR than the other methods. The comparison results for deblurring of the three methods of the four images are shown in Figure 3.

4.2. Data Classification

In this section, a learning algorithm named extreme learning machine (ELM) [29] will be investigated. ELM is a learning algorithm for single-hidden layer feedforward neural networks (SLFNs). Let $D=\{({\mathbf{x}}_i,{\mathbf{t}}_i):{\mathbf{x}}_i\in {\mathbb{R}}^n,{\mathbf{t}}_i\in {\mathbb{R}}^m,i=1,2,\dots ,N\}$ be a training dataset with N distinct training data ${\mathbf{x}}_i$ and label ${\mathbf{t}}_i$. For a given M nodes in the hidden layer, the SLFNs output for the jth pattern, ${\mathbf{o}}_j\in {\mathbb{R}}^m$, is given by:

$${\mathbf{o}}_j=\sum _{i=1}^M{\mathbf{w}}_{i}f(\langle {\mathbf{h}}_i,{\mathbf{x}}_j\rangle +b_i),j=1,2,\dots ,N,$$

(29)

where f is the activation function, ${\mathbf{h}}_i\in {\mathbb{R}}^n$ and $b_i\in \mathbb{R}$ for $i=1,2,\dots ,M$ are the weight vector and bias connecting the input layer to the ith hidden node, respectively, and ${\mathbf{w}}_i\in {\mathbb{R}}^m$ for $i=1,2,\dots ,M$ is the weight vector connecting the ith hidden layer to the output layer. The target of SLFNs is to approximate the parameters ${\mathbf{w}}_i,{\mathbf{h}}_i,b_i$ for all $i=1,2,\dots ,M$ such that:

$${\mathbf{t}}_j=\sum _{i=1}^M{\mathbf{w}}_{i}f(\langle {\mathbf{h}}_i,{\mathbf{x}}_j\rangle +b_i),j=1,2,\dots ,N,$$

(30)

which means that zero error ${\sum }_{i=1}^N∥{\mathbf{o}}_i-{\mathbf{t}}_i∥$ is close to 0 while ELM is used to find only parameter ${\mathbf{w}}_i$ with random ${\mathbf{h}}_i$ and $b_i$. As the above N equations, Equation (30) can be rewritten as:

$$\mathbf{Hw}=\mathbf{T}$$

(31)

where:

$$\mathbf{H}={\left[\begin{array}{ccc}f(\langle {\mathbf{h}}_1,{\mathbf{x}}_1\rangle +b_1)& \cdots & f(\langle {\mathbf{h}}_M,{\mathbf{x}}_1\rangle +b_M)\\ ⋮& \ddots & ⋮\\ f(\langle {\mathbf{h}}_1,{\mathbf{x}}_N\rangle +b_1)& \cdots & f(\langle {\mathbf{h}}_M,{\mathbf{x}}_N\rangle +b_M)\end{array}\right]}_{N\times M},$$

$\mathbf{w}={\left[\begin{array}{c}{\mathbf{w}}_1^T,\dots ,{\mathbf{w}}_M^T\end{array}\right]}_{m\times M}^T$ and $\mathbf{T}={\left[\begin{array}{c}{\mathbf{t}}_1^T,\dots ,{\mathbf{t}}_N^T\end{array}\right]}_{m\times N}^T.$ From Equation (31), the ELM learning algorithm estimates the weight $\mathbf{w}$ by $\mathbf{w}={\mathbf{H}}^{†}\mathbf{T}$ where ${\mathbf{H}}^{†}={\left({\mathbf{H}}^T\mathbf{H}\right)}^{-1}{\mathbf{H}}^T$ is the pseudo-inverse matrix of $\mathbf{H}$. Note that the linear system (31) can be represented by a least squared method. As shown in [29], ELM has an extremely fast training speed and good generalization performance. Nevertheless, its solutions also have some drawbacks [30]. To overcome these drawbacks, regularized extreme learning machine (RegELM) [30] replacing the least square method by the regularization method, i.e., ridge regression, for the training model was proposed, and the mathematical model of the RegELM algorithm can be described as:

$$\underset{\mathbf{w}\in {\mathbb{R}}^{M\times m}}{min}\frac{1}{2}{∥\mathbf{Hw}-\mathbf{T}∥}_2^2+\frac{\lambda }{2}{∥\mathbf{w}∥}_2^2,$$

(32)

where $\lambda >0$ is called the regularization parameter. The RegELM’s output weight can be calculated by $\mathbf{w}={(\lambda \mathbf{I}+{\mathbf{H}}^T\mathbf{H})}^{†}{\mathbf{H}}^T\mathbf{T}$, where $\mathbf{I}$ is the identity matrix. Although RegELM can be expected to provide better generalization ability than ELM and its running time is extremely fast similarly to ELM, we can define a greater generalization of RegELM as in [31] by replacing Equation (32) in a generalized way as follows:

$$\underset{\mathbf{w}\in {\mathbb{R}}^{M\times m}}{min}\frac{1}{2}{∥\mathbf{Hw}-\mathbf{T}∥}_2^2+\lambda [(1-\alpha )\frac{1}{2}{∥\mathbf{w}∥}_2^2+\alpha {∥\mathbf{w}∥}_1],$$

(33)

where $0\le \alpha \le 1$. Equation (33), called elastic net, trades off between the ride regression ($\alpha =0$) and the LASSO ($\alpha =1$). In this paper, we present a new algorithm for RegELM and employ our results to data classification problems with benchmark datasets. For this case, we set $\alpha =1$, and Problem (33) becomes a LASSO problem. From our result (Theorem 6) in Section 3, we can apply AVFBM (Algorithm 2) to solve the LASSO problem and define a learning algorithm for RegELM as follows:

RegELM-AVFBM: Given a training set $D=\{({\mathbf{x}}_i,{\mathbf{t}}_i):{\mathbf{x}}_i\in {\mathbb{R}}^n,{\mathbf{t}}_i\in {\mathbb{R}}^m,i=1,2,\dots ,N\}$, activation function f,

• Step 1: Select regularization parameter $\lambda$ and hidden node number M.
• Step 2: Randomly ${\mathbf{h}}_i$ and $b_i,i=1,\dots ,M.$
• Step 3: Calculate the hidden layer output matrix $\mathbf{H}$.
• Step 4: Obtain the output weight $\mathbf{w}$ by using AVFBM (Algorithm 2).

Several benchmark problems were chosen for experiments. All datasets were downloaded from https://archive.ics.uci.edu/ (accessed on 6 April 2020). The information of each dataset viz name of datasets, the number of attributes (number of input nodes), the number of classes (number of output nodes), and the number of (sample) data are summarized in

Table 1. Information of benchmark datasets.

Datasets	# Attributes	# Classes	# Observations
			# Train (≈70%)	# Test (≈30%)
Zoo	16	7	70	31
Iris	4	3	105	45
Wine	13	3	128	50
Parkinsons	23	2	135	60
Heart Disease UCI	14	2	213	90
Abalone	8	3	2924	1253

Note that the cardinal number of set A is denoted by #A. For example, # Attributes is the number of attributes of the data.

. Each dataset was normalized to zero mean and unit variance; 70% of the data were sampled for training, and the remaining 30% were used for testing. For each method, we tested a different number of hidden nodes M in order to see which architecture provided the best results. The number of nodes in the hidden layer varied from 1 to 200 for the abalone dataset and from 1 to 100 for the other five datasets. For each method, we set the sigmoid function as the activation function f and the regularization parameter $\lambda =1\times {10}^{-5}$ for regularized methods (RegELM and RegELM-AVFBM). However, for approximation methods (AVFBM, FISTA), we use relative error criteria, $\frac{∥x_k-x_{k-1}∥}{∥x_k∥}<ϵ$, for the stopping algorithm and set all control sequences (${\lambda }_k,{\sigma }_k,{\alpha }_k,{\beta }_k,{\gamma }_k,c_k,{\tau }_k,{\mu }_k)$ as in Section 4.1. For evaluating the performance of each method, an accuracy is defined as the total accuracy rate of classifying each case correctly. Accuracy is a value that represents the power of a model to correctly predict, and it is described as follows.

$$Accuracy=(TP+TN)/(TP+FP+TN+FN).$$

In experimental results, the accuracy of training and testing in percentage and the suitable number of hidden nodes of our method compared with direct methods viz standard ELM [29] and RegELM [30] are described in

Table 2. Comparison of the accuracy of training and testing as well as the number of hidden nodes for ELM, RegELM, and RegELM-AVFBM.

Datasets	ELM			RegELM			RegELM-AVFBM
	Training (%)	Testing (%)	# Nodes	Training (%)	Testing (%)	# Nodes	Training (%)	Testing (%)	# Nodes
Zoo	97.1429	93.5484	13	97.1429	93.5484	13	100	96.7742	93
Iris	99.0476	100	60	98.0952	100	42	98.0952	100	54
Wine	98.4375	100	36	98.4375	100	36	100	100	40
Parkinsons	94.8148	75	31	94.8148	75	31	96.2963	81.6667	78
Heart Disease UCI	86.385	84.4444	25	86.385	84.4444	25	88.7324	85.5556	33
Abalone	69.0492	67.4381	89	69.0492	67.4381	89	68.4337	67.518	111

. RegELM-AVFBM has a good behavior in terms of accuracy of prediction and fit for testing datasets compared with the two direct methods. However, it is hard to compare the computational time, since approximation methods take time to iterate for convergence to the solution. Thus, to evaluate in the same way, we use two approximation methods (FISTA and AVFBM) for training RegELM and train the model with five different stopping errors $ϵ$ under a maximum of 100,000 iterations.

Table 3. Comparison of the accuracy of training and testing, computation time, number of iterations, and number of hidden nodes for RegELM-FISTA and RegELM-AVFBM. The sign ∞ in the column # Iters means that the model was computed over the maximum iterations (100,000 iterations for this case).

Datasets	$\mathit{ϵ}$	RegELM-FISTA					RegELM-AVFBM
		Training (%)	Testing (%)	Time(s)	# Iters	# Nodes	Training (%)	Testing (%)	Time(s)	# Iters	# Nodes
Zoo	0.1	90	90.3226	0.0005994	11	33	98.5714	93.5484	0.0008407	9	82
	0.01	98.5714	93.5484	0.0021261	40	72	100	96.7742	0.0079208	81	93
	0.001	100	93.54839	0.0085405	455	42	98.57143	93.54839	0.005203	157	13
	0.0001	97.142857	93.548387	0.0152656	1330	13	97.14286	93.54839	0.0142225	384	13
	0.00001	97.142857	93.548387	0.0353528	2609	13	97.14286	93.54839	0.0193836	694	13
	0.000001	97.142857	93.548387	0.0561337	4193	13	97.142857	93.548387	0.0317928	1354	13
Iris	0.1	79.0476	91.1111	0.0006506	11	39	80	91.1111	0.0003463	7	20
	0.01	80	91.1111	0.0008544	27	9	96.19048	100	0.0033075	110	56
	0.001	92.38095	97.77778	0.0141314	658	22	98.09524	100	0.0217391	804	54
	0.0001	96.190476	100	0.051715	4232	38	98.095238	100	0.1359273	4891	53
	0.00001	98.0952381	100	0.6242778	41,584	56	98.0952381	100	0.9981745	47,695	42
	0.000001	-	-	-	∞	-	-	-	-	∞	-
Wine	0.1	97.6563	96	0.0005292	11	31	99.2188	98	0.0008743	8	65
	0.01	99.2188	98	0.0015979	30	64	100	100	0.0047111	98	40
	0.001	99.21875	100	0.0106758	364	45	98.4375	100	0.006298	271	36
	0.0001	98.4375	100	0.0536025	4374	36	98.4375	100	0.0234622	1146	36
	0.00001	98.4375	100	0.2150904	18,406	36	98.4375	100	0.0710794	3135	36
	0.000001	98.4375	100	0.4733094	39,108	36	98.4375	100	0.1426151	7342	36
Parkinsons	0.1	80.7407	75	0.0005362	11	5	80.7407	73.3333	0.0007843	4	5
	0.01	80.7407	75	0.000316	11	5	96.2963	81.66667	0.0034927	111	78
	0.001	96.2963	78.33333	0.0143303	649	83	95.55556	76.66667	0.0092722	252	31
	0.0001	98.518519	85	0.1072512	4702	95	100	76.666667	0.0401135	1533	60
	0.00001	99.2592593	78.3333333	0.3693551	31,488	60	95.555556	75	0.0395779	2266	31
	0.000001	95.5555556	75	0.3067227	32,185	31	94.814815	75	0.0887627	5421	31
Heart Disease UCI	0.1	82.6291	84.4444	0.000561	11	52	86.8545	85.5556	0.0006593	8	72
	0.01	84.9765	84.4444	0.0008177	31	57	85.9155	84.4444	0.0027626	73	25
	0.001	87.79343	86.66667	0.0115745	600	61	88.73239	85.55556	0.0051466	240	33
	0.0001	90.140845	85.555556	0.1300254	5231	58	86.38498	84.44444	0.0131317	644	25
	0.00001	86.384977	84.444444	0.0629385	6507	25	86.384977	84.444444	0.042063	1707	25
	0.000001	86.3849765	84.4444444	0.1245222	12,505	25	86.384977	84.444444	0.054982	3366	25
Abalone	0.1	57.2845	56.3448	0.0008332	11	9	57.0109	56.664	0.0007203	7	16
	0.01	59.13133	57.86113	0.0116477	47	147	66.72367	66.0016	0.0199067	111	96
	0.001	64.74008	64.08619	0.07772	445	111	68.63885	67.11891	0.2978755	817	175
	0.0001	66.792066	66.400638	0.8201147	5560	96	68.433653	67.517957	0.9515634	4480	111
	0.00001	68.5362517	67.1987231	11.9269803	51,900	149	68.7414501	67.6775738	3.4826104	21,877	89
	0.000001	-	-	-	∞	-	68.9466484	67.6775738	13.0566531	81,392	89

shows the performance viz accuracy of training and testing (in percentage), computational time (in second), number of computed iterations, and number of suitable nodes in the hidden layer.

5. Conclusions

In this work, by using the inertial technique together with the viscosity approximation method, we propose a new accelerated algorithm for finding a common fixed point of a countable family of nonexpansive operates in a real Hilbert space. The strong convergence of the proposed method is established under some suitable conditions. As a special case, we obtain a new accelerated algorithm, called the accelerated viscosity forward–backward method (AVFBM), for solving nonsmooth convex optimization problems. We also apply our algorithm, AVFBM, to solving image restoration and classification problems. By our experiments, for image restoration problem, they show that our algorithm, AVFBM, has a better performance for SNR and PSNR than that of FBS and FISTA, which are the most popular methods for solving such problems. Moreover, for the classification problems of six datasets—Zoo, Iris, Wine, Parkinsons, Heart Disease UCI, and Abalone (https://archive.ics.uci.edu/, accessed on 6 April 2020)—we use our algorithm, AVFBM, as a learning algorithm for finding the optimal output weight $\mathbf{w}$ in the mathematical model (32) of the classification problems. We compare the efficiency of our method with ELM, RegELM, and RegELM-FISTA by using the measurement of accuracy of training and testing. We found that our algorithm outperforms the other methods, as seen from Table 2 and Table 3.