Kernel Ridge Regression Model Based on Beta-Noise and Its Application in Short-Term Wind Speed Forecasting

Abstract

The Kernel ridge regression ($KRR$) model aims to find the hidden nonlinear structure in raw data. It makes an assumption that the noise in data satisfies the Gaussian model. However, it was pointed out that the noise in wind speed/power forecasting obeys the Beta distribution. The classic regression techniques are not applicable to this case. Hence, we derive the empirical risk loss about the Beta distribution and propose a technique of the kernel ridge regression model based on the Beta-noise ($BN$-$KRR$). The numerical experiments are carried out on real-world data. The results indicate that the proposed technique obtains good performance on short-term wind speed forecasting.

1. Introduction

Linear regression ($LR$) is an approach to using the least squares method to model the relationship between a scalar dependent variable and one or more explanatory variables. It also refers to the plane points that are fitted with a straight line or the points in a high dimension space that are fitted with a hyperplane. This method is very sensitive to predictors being in a configuration of near-collinearity. Ridge regression ($RR$) is a variant of linear regression whose goal is to circumvent the problem of predictors collinearity. The ridge regression model is a powerful technique of machine learning which was introduced by Hoerl [1] and Hastie et al. [2], and it is a method from classical statistics that implements a regularized form of least squares regression [3]. Ridge regression is an alternative method for learning function based on a regularized extension of least squares techniques [4].

Given the data-set

$$D_N=(x_1,y_1),(x_2,y_2),\dots ,(x_N,y_N)$$

(1)

where $x_i\in X=R^n$, $y_i\in R$, $i=1,\dots ,N$ is the data-set. A multiple $LR$ is $f(x)={\varpi }^T·x+b$. R represents real number set, $R^n$ is n dimensional Euclidean space, N is the number of sample points, superscript T denotes the matrix transpose. $LR$ and $RR$ determine the parameter vector $\varpi \in R^n$ by minimizing the objective functions, respectively:

$$g_{LR}=\sum _{i=1}^N{(y_i-{\varpi }^T·x-b)}^2,$$

(2)

$$g_{RR}=\frac{1}{2}·{\varpi }^T·\varpi +C·\sum _{i=1}^N{(y_i-{\varpi }^T·x-b)}^2.$$

(3)

The objective function used in ridge regression implements a form of Tikhonov [5] regularization of a sum-of-squares error metric, which is a regularization parameter controlling the bias-variance trade-off [6]. This corresponds to penalized maximum likelihood estimation of $\varpi$, assuming the targets have been corrupted by independent identical probability distribution (i.i.d.) samples from a Gaussian noise process with zeros mean and variance ${\sigma }^2$, i.e., $y_i=f(x_i)+{\xi }_i,\xi \in N(0,{\sigma }^2),i=1,\dots ,N$.

The $KRR$ model based on Gaussian-noise characteristic is derived by Saunders et al [7]. $RR$ [1,3,5] aims to find the hidden nonlinear structure in the raw data, while nonlinear mapping is approximated by means of $KRR$ based on kerneltechniques [7,8,9,10,11]. Therefore a linear $RR$ model is constructed in a feature-space H ($\mathsf{\Phi }:R^n⟶H$), induced by a nonlinear kernel function defining the inner product $K(x_i,x_j)=(\mathsf{\Phi }(x_i)·\mathsf{\Phi }(x_j))$ ($i,j=1,\dots ,N$). The kernel function $\mathsf{\Phi }:R^n\to H$ may be any positive definite Mercer kernel. Therefore, the objective function of $KRR$ based on Gaussian-noise ($GN$-$KRR$) minimization can be written as

$$g_{GN-KRR}=\frac{1}{2}·{\varpi }^T·\varpi +C·\sum _{i=1}^N{(y_i-{\varpi }^T·\mathsf{\Phi }(x)-b)}^2.$$

(4)

Suppose the noise is Gaussian, the $GN-KRR$ model may meet the requirements. However, the noise in wind speed and wind power forecast does not obey the Gaussian distribution, but the Beta distribution. The classic regression techniques are not applicable to above case. The uncertainty of wind power predictions was investigated in [12]. The statistics of the wind power forecasting error were not Gaussian. The work in [13] also found that the output of wind turbine systems is limited between zero and the maximum power and the error statistics do not follow a normal distribution. It also proved that using the Beta-function is justifiable for wind power prediction about chi-squared tests. In [14], the standard deviation of the data set was a function of the normalized predicted power $p=p_{pred}/p_{inst}$, where $p_{pred}$ is the predicted power and $p_{inst}$ is the wind power installed capacity. Fabbri [14] pointed out standardized production power p be within the interval $[0,1]$ and Beta-function are more suitable than standard normal distribution. Literature [15] exhibited the advantages of using Beta-probability distribution function (pdf) instead of Gaussian pdf for approximating the forecasting error. Based on the above literature [12,13,14,15,16], this work plans to study the error of Beta-distribution between the predicted values $x_p$ and the measured values $x_m$ in the wind speed forecasting, and pdf of ${\epsilon }_i$ is $f({\epsilon }_i)={{\epsilon }_i}^{u-1}·{(1-{\epsilon }_i)}^{v-1}·h,{\epsilon }_i\in (0,1),i=1,2,\cdots ,N$, plotted below in Figure 1. Where $u,v$ are parameters, h is normalization-factor, and the parameters $u,v$ may be determined by the given values of mean and standard deviation [17].

It is not suitable to apply the $KRR$ techniques based on Gaussian-noise model ($GN$-$KRR$) to fit functions from data-set with Beta-noise. In order to solve the above problem, this work focuses on the utilization of optimization theory and Beta-noise loss function and derives a method of $KRR$ based on Beta-noise characteristic ($BN$-$KRR$). It also introduces a forecasting technique that can deal with high-dimensionality and nonlinearity simultaneously.

This paper is organized as follows. In Section 2, we will derive the Beta-noise empirical risk loss by the Bayesian principle. Section 3 describes the proposed $KRR$ model based on the Beta-noise. Section 4 gives the solution and algorithm design of $KRR$ of the Beta-noise characteristic based on Genetic Algorithm. The numerical experiments are carried out on $BN$-$KRR$ to short-term wind speed and wind power prediction in Section 5, respectively. Finally, the conclusions and future work are given in Section 6.

2. Bayesian Principle to Beta-Noise Empirical Risk Loss

Learning to fit data with noise is an important problem in many real-world data mining applications. Given a training set $D_N$ of (1) with noise is additive

$$y_i=f(x_i)+{\xi }_i,i=1,\cdots ,N$$

(5)

where ${\xi }_i$ is random i.i.d. $P({\xi }_i)$ with standard deviation $\sigma$ and mean $\mu$.

The objective is to find regressor f minimizing the expected risk [18,19] $R\left[f\right]=\int l(x,y,f(x))dP(x,y)$ based on the empirical data $D_l$, where $l(x,y)$ is a empirical risk loss (determining how we will penalize estimation errors). Since we do not know the distribution $P(x,y)$, it can only use data-set $D_N$ to estimate a regressor f and minimize $R\left[f\right]$. A possible approximation consists of replacing the integration by the empirical estimate to get the empirical risk $R_{emp}\left[f\right]=\frac{1}{N}·{\sum }_{i=1}^{N}l(x,y_i,f(x_i))$. In general, we should add a capacity control term in $RR$ and $KRR$, which leads to the regularized risk functional [18,20]

$$R_{reg}\left[f\right]=R_{emp}\left[f\right]+\frac{\lambda }{2}·{\parallel \varpi \parallel }^2,$$

(6)

where $\lambda >0$ is a regularization constant, $R_{emp}\left[f\right]$ is the empirical risk. It is well known that $l({\xi }_i)=\frac{1}{2}·{\xi }_i^2$ is the empirical risk loss of Gaussian-noise characteristics for $LR$ (2), $RR$ (3), and $KRR$ (4). However, what is the empirical risk loss about Beta-noise of $KRR$ model? The Beta-noise empirical risk loss by the use of Bayesian principle is given as follows.

The regressor $f(x)$ is unknown, the objection is to estimate the regressor $f(x)$ from $g\in D_N$. According to the literature [20,21,22], the optimal empirical risk loss from maximum likelihood be

$$l(x,y,f(x))=-logp(y-f(x)).$$

(7)

The maximum likelihood estimation be

$$X_f=\{(x_1,f(x_1)),(x_2,f(x_2)),\dots ,(x_N,f(x_N))\},$$

(8)

$$p(X_f|X)=\prod _{i=1}^{N}p(f(x_i)|(x_i,y_i))=\prod _{i=1}^{N}p(y_i-f(x_i)).$$

(9)

Maximizing $p(X_f|X)$ is equivalent to minimizing $-log(p(X_f|X))$. Using Equation (7), we have

$$l(x,y,f(x))=-logp(X_f|X).$$

(10)

Suppose noise in Equation (5) adheres to Beta distribution with mean $\mu \in (0,1)$ and variance ${\sigma }^2$, thus we can get $u=(1-\mu )·{\mu }^2/{\sigma }^2-\mu$, $v=(1-\mu )/\mu ·u$ [13,14], where $h=\mathsf{\Gamma }(u+v)/\mathsf{\Gamma }(u)·\mathsf{\Gamma }(v)$ is the normalization-factor. By Equation (10), the Beta-noise empirical risk loss is

$$l(\xi )=l(y-f(x))=(1-u)log(\xi )+(1-v)log(1-\xi ).$$

(11)

Empirical risk loss of Gauss-noise and Beta-noise with different parameters is shown in Figure 2.

3. $\mathit{KRR}$ Model Based on Beta-Noise

It is not appropriate to apply the $KRR$ model based on Gaussian-noise characteristic ($GN$-$KRR$) to deal with tasks with the Beta-noise distribution. Consequently, we use Beta-noise loss function and maximum likelihood method to estimate the optimal loss function. Now, we derive the optimal empirical risk loss about Beta-noise distribution, and propose a new technique of the $KRR$ model based on Beta-noise characteristic ($BN$-$KRR$).

First, considering constructing $LR$ regressor $f(x)={\varpi }^T·x_i+b$, where $x_i={(1,x_{i1},\dots ,x_{in})}^T$ ($i=1,2,\cdots ,N$), $\varpi ={({\varpi }_0,{\varpi }_1,\cdots ,{\varpi }_n)}^T$. We use kernel techniques and construct the kernel function $K(•,•)$ where $K(x_i,x_j)=(\mathsf{\Phi }(x_i)·\mathsf{\Phi }(x_j))$, $\mathsf{\Phi }:R^n\to H$, H is Hilbert space, and $(x_i·x_j)$ ($i,j=1,2,\cdots ,N$) is inner product of H. Then we extend kernel techniques to the ridge regression model based on the Beta-noise characteristic.

Let the set of inputs be $\{(x_i,y_i),i=1,\cdots ,N\}$, where i represents the indicator for the i-th sample in $D_l$. For the general Beta-noise characteristic, it is Formula (11) that the Beta-noise loss function $c({\xi }_i)$ in the sample point $\{(x_i,y_i)\}$ of $D_N$. Owing to the fact that ridge regression and $KRR$ techniques with Gaussian-noise characteristic ($GN-KRR$) are not suitable to Beta-noise distribution in time series problems, the Formula (11) is selected as Beta empirical risk loss to overcome the shortage of $GN$-$KRR$. The primal problem of $KRR$ model with the Beta-noise (Denoted by $BN$-$KRR$) can be described as follows ($C>0$)

$$\begin{array}{c}{\displaystyle min\{g_{P_{BN-KRR}}=\frac{1}{2}{\varpi }^T·\varpi +C·\sum _{i=1}^{N}c({\xi }_i)\}}\hfill \\ P_{BN-KRR}:s.t.y_i-{\varpi }^T·\mathsf{\Phi }(x_i)-b={\xi }_i.\hfill \end{array}$$

(12)

where $c({\xi }_i)=(1-u)log({\xi }_i)+(1-v)log(1-{\xi }_i)$, ${\xi }_i=y_i-f(x_i)i=1,\cdots ,N$.

Theorem 1.

Model $BN$-$KRR$’s Solution to original Problem (12) about ϖ exists and is unique.

Proof. 

The existence of solutions is trivial. The uniqueness of solutions is shown below. If we have solutions $\overline{\varpi }$, $\tilde{\varpi }$, then Problem (12) exist solutions $(\overline{\varpi },\overline{b},\overline{\xi })$ and $(\tilde{\varpi },\tilde{b},\tilde{\xi })$. Define $(\varpi ,b,\xi )$ as follows

$$\varpi =\frac{1}{2}(\overline{\varpi }+\tilde{\varpi }),b=\frac{1}{2}(\overline{b}+\tilde{b}),\xi =\frac{1}{2}(\overline{\xi }+\tilde{\xi }).$$

(13)

We have

$$y_i-{\varpi }^T·\mathsf{\Phi }(x_i)-{\xi }_i=y_i-\frac{1}{2}{(\overline{\varpi }+\tilde{\varpi })}^T·\mathsf{\Phi }(x_i)-\frac{1}{2}(\overline{b}+\tilde{b})-\frac{1}{2}(\overline{\xi }+\tilde{\xi })$$

$$=\frac{1}{2}(y_i-{\overline{\varpi }}^T·\mathsf{\Phi }(x_i)-\overline{b}-{\overline{\xi }}_i)+\frac{1}{2}(y_i-{\tilde{\varpi }}^T·\mathsf{\Phi }(x_i)-\tilde{b}-{\tilde{\xi }}_i)=0,$$

where $\frac{1}{2}(\overline{\xi }+\tilde{\xi })\ge 0(i=1,\cdots ,l)$, $(\varpi ,b,\xi )$ is a feasible solution of Problem (12). Further,

$$\frac{1}{2}{\parallel \varpi \parallel }^2+C\sum _{i=1}^{N}c({\xi }_i)\ge \frac{1}{2}{\parallel \overline{\varpi }\parallel }^2+C\sum c({\tilde{\xi }}_i),$$

(14)

$$\frac{1}{2}{\parallel \varpi \parallel }^2+C\sum _{i=1}^{N}c({\xi }_i)\ge \frac{1}{2}{\parallel \tilde{\varpi }\parallel }^2+C\sum c({\tilde{\xi }}_i).$$

(15)

By Inequalities (14) and (15), we get ${2\parallel \varpi \parallel }^2\ge \parallel \overline{\varpi }{\parallel }^2+{\parallel \tilde{\varpi }\parallel }^2$. $\varpi =\overline{\varpi }+\tilde{\varpi }$ is substituted into the above inequality

$$\parallel \overline{\varpi }+\tilde{\varpi }{\parallel }^2\ge 2(\parallel \overline{\varpi }{\parallel }^2+\parallel \tilde{\varpi }{\parallel }^2),$$

(16)

as $\parallel \overline{\varpi }+\tilde{\varpi }\parallel \le \parallel \overline{\varpi }\parallel +\parallel \tilde{\varpi }\parallel$, then

$$(\parallel \overline{\varpi }\parallel +\parallel \tilde{\varpi }{\parallel )}^2\ge 2(\parallel \overline{\varpi }{\parallel }^2+\parallel \tilde{\varpi }{\parallel }^2).$$

(17)

In addition, $2\parallel \overline{\varpi }\parallel ·\parallel \tilde{\varpi }\parallel \le \parallel \overline{\varpi }{\parallel }^2+{\parallel \tilde{\varpi }\parallel }^2$, by (17), get

$$2\parallel \overline{\varpi }\parallel ·\parallel \tilde{\varpi }\parallel =\parallel \overline{\varpi }{\parallel }^2+\parallel \tilde{\varpi }{\parallel }^2,\parallel \overline{\varpi }\parallel =\parallel \tilde{\varpi }\parallel ,\parallel \overline{\varpi }+\tilde{\varpi }\parallel =\parallel \overline{\varpi }\parallel +\parallel \tilde{\varpi }\parallel .$$

For $\tilde{\varpi }=m·\overline{\varpi }$, thus $m=1$ or $m=-1$. Since $m=-1$, then $\overline{\varpi }+\tilde{\varpi }=0$. By $\parallel \overline{\varpi }+\tilde{\varpi }\parallel =\parallel \overline{\varpi }\parallel +\parallel \tilde{\varpi }\parallel$, get $\parallel \overline{\varpi }\parallel =\parallel \tilde{\varpi }\parallel =0$. Namely, $\overline{\varpi }=\tilde{\varpi }=0$. For $m=1$, thus $\overline{\varpi }=\tilde{\varpi }$.

In conclusion, Solution to Problem (12) exists and is unique. □

Theorem 2.

Model $BN$-$KRR$’s dual Problem of primal Problem (12) is

$$\begin{array}{c}{\displaystyle Max\{g_{D_{BN-KRR}}=-\frac{1}{2}\sum _{i=1}^N\sum _{j=1}^N({\alpha }_i·{\alpha }_j·K(x_i,x_j))+\sum _{i=1}^N({\alpha }_i·y_i)}\hfill \\ +C\sum _{i=1}^N((1-u)log({\xi }_i({\alpha }_i))+(1-v)log(1-{\xi }_i({\alpha }_i)))\}\hfill \\ D_{BN-KRR}:s.t.\sum _{i=1}^l({\alpha }_i)=0\hfill \end{array}$$

(18)

where ${\xi }_i({\alpha }_i)=\frac{2+{\alpha }_i/C-u-v-{\Delta }^{\frac{1}{2}}}{2·{\alpha }_i/C}$, $\Delta ={({\alpha }_i/C+u-v)}^2+4·(1+u·v-u-v)$ ($i=1,\cdots ,N$) and $C>0$ is constant.

Proof. 

The introduction of Lagrange functional $L(\varpi ,b,\alpha ,\xi )$ is

$$L(\varpi ,b,\alpha ,\xi )=\frac{1}{2}{\varpi }^T·\varpi +C·\sum _{i=1}^N(c({\xi }_i))+\sum _{i=1}^N{\alpha }_i(y_i-{\varpi }^T·\mathsf{\Phi }(x_i)-b-{\xi }_i).$$

For the sake of the minimum $L(\varpi ,b,\alpha ,\xi )$, seek partial derivative to $\varpi ,b,\alpha ,\xi$ respectively. From Karush-Kuhn-Tucker (KKT) conditions, obtain

$${\nabla }_{\varpi }(L)=0,{\nabla }_b(L)=0,{\nabla }_{\xi }(L)=0.$$

So $\varpi ={\sum }_{i=1}^N({\alpha }_i·\mathsf{\Phi }(x_i)),{\sum }_{i=1}^N({\alpha }_i)=0,C·\frac{\partial (c({\xi }_i))}{\partial {\xi }_i}-{\alpha }_i=0.$

Substituting the extreme conditions into $L(\varpi ,b,\alpha ,\xi )$ and finding the Maximum of α, thus derive the dual Problem (18) of Problem (12). □

On account of $\frac{\partial (c({\xi }_i))}{\partial ({\xi }_i)}=\frac{1-\alpha }{{\xi }_i}-\frac{1-\beta }{1-{\xi }_i}$, by $C·\frac{\partial (c({\xi }_i))}{\partial ({\xi }_i)}-{\alpha }_i=0$, we have ${\alpha }_i/C·{\xi }_i^2-(2+{\alpha }_i/C-\alpha -\beta )·{\xi }_i+1-\alpha =0.$

Now get

$${\xi }_{i1}({\alpha }_i)=\frac{2+{\alpha }_i/C-\alpha -\beta +{\Delta }^{\frac{1}{2}}}{2·{\alpha }_i/C},{\xi }_{i2}({\alpha }_i)=\frac{2+{\alpha }_i/C-\alpha -\beta -{\Delta }^{\frac{1}{2}}}{2·{\alpha }_i/C}.$$

where $\Delta ={({\alpha }_i/C+\alpha -\beta )}^2+4·(1+\alpha ·\beta -\alpha -\beta )$. Because of $0<{\xi }_i({\alpha }_i)<1$, we reject ${\xi }_{i1}({\alpha }_i)$ and let ${\xi }_i({\alpha }_i)={\xi }_{i2}({\alpha }_i)$.

We have $\varpi ={\sum }_{i=1}^N({\alpha }_i·\mathsf{\Phi }(x_i)),b=y_i-{\sum }_{i=1}^N({\alpha }_j·K(x_j,x_i))-{\xi }_i({\alpha }_i)$, gain the decision-making function of $BN$-$KRR$ is

$$f(x)={\varpi }^T·\mathsf{\Phi }(x)+b=\sum _{i=1}^N({\alpha }_i·K(x_i,x))+b.$$

Note: The $KRR$ of the Gaussian-noise characteristic ($GN-KRR$) was discussed in [9,10,11]. The Gaussian empirical risk loss in the sample point $(x_i,y_i)\in D_N$ is $c({\xi }_i)=\frac{1}{2}{\xi }_i^2$, thus the dual Problem of model $KRR$ based on Gaussian-noise characteristic ($GN-KRR$) is

$$\begin{array}{c}{\displaystyle max\{g_{D_{GN-KRR}}=-\frac{1}{2}\sum _{i=1}^N\sum _{j=1}^N({\alpha }_i·{\alpha }_j·K(x_i,x_j))+\sum _{i=1}^N({\alpha }_i·y_i)-\frac{1}{2C}·\sum _{i=1}^N({\alpha }_i^2)\}}\hfill \\ D_{GN-KRR}:s.t.\sum _{i=1}^N{\alpha }_i=0.\hfill \end{array}$$

(19)

The dual Problem of model $RR$ based on the Gaussian-noise characteristic ($GN$-$RR$) is

$$\begin{array}{c}{\displaystyle max\{g_{D_{GN-RR}}=-\frac{1}{2}\sum _{i=1}^N\sum _{j=1}^N({\alpha }_i·{\alpha }_j·(x_i,x_j))+\sum _{i=1}^N({\alpha }_i·y_i)-\frac{1}{2C}·\sum _{i=1}^N({\alpha }_i^2)\}}\hfill \\ D_{GN-RR}:s.t.\sum _{i=1}^N{\alpha }_i=0.\hfill \end{array}$$

(20)

4. Solution Based on Genetic Algorithm

We get the Solution and algorithm design of model $KRR$ based on Beta-noise characteristic ($BN$-$KRR$) as follows.

(1)

Let training samples $D_N=\{(x_1,y_1),(x_2,y_2),\dots ,(x_N,y_N)\}$, where $x_i\in X=R^n$, $y_i\in R$ ($i=1,\dots ,N$).

(2)

Select the appropriate positive $C,u,v$ and the suitable kernel $K(•,•)$.

(3)

Solve optimization Problem (18), gain optimal Solution $\alpha =({\alpha }_1,\cdots ,{\alpha }_N)$.

(4)

Construct the decision-making function

$$f(x)={\varpi }^T·\mathsf{\Phi }(x)+b=\sum _{i=1}^N({\alpha }_i·K(x_i,x))+b,$$

and $b=y_i-{\sum }_{i=1}^N({\alpha }_j·K(x_j,x_i))-{\xi }_i({\alpha }_i)$.

The confirmation of unknown parameters of model $BN-KRR$ is a complicated process and the appropriate parameter combination of the models can enhance the regression accuracy of the kernel ridge regression based on Beta-noise. Genetic Algorithm (GA) [23,24,25] is a search heuristic that mimics the process of natural evolution, this heuristic is routinely used to generate useful solutions to optimize and search problems. In GA, the evolution usually starts from a population of randomly generated individuals and happens in generations. In each generation, the fitness of every individual in the population is evaluated, multiple individuals are stochastically selected from the current population and modified to form a new population. The new population is then used in the next iteration of the algorithm. Commonly, the algorithm terminates when either a maximum number of generations has been produced, or a satisfactory fitness level has been reached for the population. If the algorithm has terminated due to a maximum number of generations, a satisfactory solution may or may not have been reached.

GA is considered as one of the modern optimization algorithms to solve the combinatorial optimization problem and is used to determine the parameters of model $BN$-$KRR$. Based on the survival and reproduction of the fitness, GA is continually applied to get new and better solutions without any pre-assumptions, such as continuity and unimodality [26,27,28]. The proposed model $BN$-$KRR$ has been implemented in Matlab 7.8 programming language. The experiments are made on the 8.0 GHz Core (TM) i7-4790 CPU personal computer with 3.60 GB memory under Microsoft Windows XP Professional. The initial parameters of GA are $Max-cgen=100$, $C\in [1,201]$, $u,v\in (0,\propto )$. Many practical applications display that polynomial and Gaussian kernels perform well under general smooth assumptions [29]. This work, polynomial, and Gaussian kernels can be used as the kernel for models $\nu$-$SVR$, $GN$-$KRR$, and $BN$-$KRR$:

$$K(x_i,x_j)={((x_i·x_j)+1)}^d,K(x_i,x_j)=exp(\frac{-\parallel x_i-x_j{\parallel }^2}{{\sigma }^2}),$$

where d is positive integer, and let $d=1,2$, or 3. $\sigma$ is positive, and take $\sigma =0.2$.

As we all know, no prediction model forecasts perfectly. There are also certain criteria, such as mean absolute error (MAE), the root mean square error (RMSE), mean absolute percentage error (MAPE), and standard error of prediction (SEP) are used to evaluate the predictive performance of models $\nu$-$SVR$, $GN$-$KRR$, and $BN$-$KRR$. The four criteria are defined as follows:

$$\mathrm{MAE}=\frac{1}{N}·\sum _{i=1}^N|x_{p,i}-x_{m,i}|,$$

(21)

$$\mathrm{MAPE}=\frac{1}{N}·\sum _{i=1}^N\frac{|x_{p,i}-x_{m,i}|}{x_{m,i}},$$

(22)

$$\mathrm{RMSE}=\frac{1}{N}·\sqrt{\sum _{i=1}^N{(x_{p,i}-x_{m,i})}^2},$$

(23)

$$\mathrm{SEP}=\frac{\sqrt{{\sum }_{i=1}^N{(x_{p,i}-x_{m,i})}^2}}{{\sum }_{i=1}^N{x}_{m,i}},$$

(24)

where l is the size of the selected samples, $m,i$ is the measured result of data-point $x_i$, and $p,i$ is the predictive result of data-point $x_i$ ($i=1,2,\cdots ,N$) [14,15,16].

5. Short-Term Wind Speed and Wind Power Forecasting with Real Data-Set

The model $BN$-$KRR$ is applied to the multi-factors actual data-set for wind speed sequence prediction from Jilin Province. The wind speed data contain more than a year of samples which are collected in intervals of ten minutes, and the number of wind speed data is 62,466. Each column attribute is mean, variance, minimum, and maximum, respectively. The short-term wind speed forecast is studied as follows.

Suppose the training sample number is 2160 (from 1 to 2160 for 15 days), and the number of test samples is 720 (from 2161 to 2880 for 5 days). The input vector is $\overrightarrow{x_i}=(x_i,x_{i+1},x_{i+2},\cdots ,$$x_{i+11})$, the output value is $x_{i+11+step}$, and $step=1,3$. Namely, the pattern above is used to forecast the wind speed each interval of 10 and 30 min at each Point $x_{i+11}$, respectively [30,31].

1. Forecast wind speed at point $x_{i+11}$ each interval of 10 min

The short-term wind speed sequence forecast results at point $x_{i+11}$ each interval of 10 min given by $GN$-$KRR$ [7,8,32], $\nu$-$SVR$ [33,34], and $BN$-$KRR$ are illuminated with Figure 3. In $GN$-$KRR$, parameter $C=151$. In $\nu$-$SVR$, parameter $C=151,\nu =0.54$. In $BN$-$KRR$, parameters $C=181,u=3.6084,$$v=3.0889$.

MAE, MAPE, RMSE, and SEP indicators are used to evaluate the prediction results of the three models at point $x_{i+11}$ each interval of 10 min shown in

Table 1. Error statistic of three models (step = 1).

Model	MAE	RMSE	MAPE(%)	SEP(%)
$\nu$-SVR	0.4280	0.5833	7.02	7.02
GN-KRR	0.4219	0.5768	7.94	7.06
BN-KRR	0.3668	0.4233	6.84	5.23

.

2. Forecast wind speed at point $x_{i+11}$ each interval of 30 min

The short-term wind power sequence forecast results at point $x_{i+11}$ each interval of 30 min given by $GN$-$KRR$, $\nu$-$SVR$, and $BN$-$KRR$ are illuminated with Figure 4. In $GN-KRR$, parameter $C=151$. In $\nu$-$SVR$, parameter $C=151,\nu =0.54$. In $BN$-$KRR$, parameters $C=181,u=3.6084,$$v=3.0889$.

MAE, MAPE, RMSE, and SEP indicators are used to evaluate the prediction results of the three models at point $x_{i+11}$ each interval of 30 min shown in

Table 2. Error statistic of three models (step = 3).

Model	MAE	RMSE	MAPE(%)	SEP(%)
$\nu$-SVR	0.7979	1.0116	23.36	12.53
GN-KRR	0.7109	0.9226	17.17	11.43
BN-KRR	0.6640	0.8417	18.82	10.43

.

The results of wind speed forecasting experiments indicate that $BN$-$KRR$ has better performance than $GN$-$KRR$ and $\nu$-$SVR$ in 10-min and 30-min short-term wind speed forecasting.

We have predicted the short-term wind speed from the Jilin Province wind farm, so we can calculate the wind power according to the Formula (25):

$$P_M=\left\{\begin{array}{cc}0,\hfill & v<v_{cut-in},\mathrm{or}v>v_{cut-out}\hfill \\ \frac{v-v_{cut-in}}{v_r-v_{cut-in}},\hfill & v_{cut-in}\le v<v_r\hfill \\ P_r,\hfill & v_r\le v<v_{cut-in}.\hfill \end{array}\right.$$

(25)

where $v_{cut-in}$ and $v_{cut-out}$ represent cut-in wind speed and cut-out wind speed of wind turbine, respectively. $v_r$ and $P_r$ represent rated wind speed and rated power of wind turbine, respectively. The predictive wind speed is substituted into the Formula (25), we can obtain the predicted wind power.

6. Conclusions and Future Work

In this work, we propose a new version of kernel ridge regression model based on the Beta-noise ($BN$-$KRR$) to predict the uncertainty system of Beta-noise. Novel results have been obtained by the use of the model $BN$-$KRR$, which takes the Bayesian principle to Beta-noise empirical risk loss and improves the prediction accuracy. The numerical experiments are carried out on real-world data (the short-term wind speed). Comparing the model $BN$-$KRR$ and models $GN$-$KRR$ and $\nu$-$SVR$ by criteria $MAE$, $MAPE$, $RMSE$, and $SEP$ verifies the validity and feasibility of our proposed model $BN-KRR$. Further, the forecasting results indicate that the proposed technique can obtain good performance on short-term wind speed forecasting.

In practical regression problems, data uncertainty is inevitable. The observed data are usually described in linguistic levels or ambiguous metrics, like the weather forecast, the forecast results of dry and wet, or sunny and cloudy, and so on. We should consider developing fuzzy kernel ridge regression algorithms with different noise models.

We verify the validity and feasibility of the model.