正则化参数求解方法研究

Research on Method of Regularization Parameter Solution

摘要:

针对BP神经网络算法训练过程中出现的过拟合问题,提出了利用一阶原点矩,二阶原点矩,方差和极大似然估计概念的推广来计算L2正则化中正则化参数λ值的方法。该方法通过对算法数据集[X,Y]中的X矩阵进行运算得到的四个λ值,BP神经网络算法训练时通常采用的是贝叶斯正则化方法,贝叶斯正则化方法存在着对先验分布和数据分布依赖等问题,而利用上述概念的推广计算的参数代入L2正则化的方法简便没有应用条件限制;在BP神经网络手写数字识别的实验中,将该方法与贝叶斯正则化方法应用到实验中后的算法识别结果进行比较,正确率提高了1.14-1.50个百分点;因而计算得到的λ值应用到L2正则化方法与贝叶斯正则化方法相比更能使得BP神经网络算法的泛化能力强,证明了该算法的有效性。

Abstract:

Aiming at the over fitting problem of BP neural network algorithm, this method is put forward to calculate the value of the regularization parameter applied to L2 regularization by using the concept of the first order origin moment, the two order origin moment, the variance and the maximum likelihood estimation.This method based on the X matrix of data sets to compute four value.Compared with the Bayesian regularization method used to train the BP neural network, Bayesian regularization method is depend on the prior distribution and the distribution of data dependence. The calculation method in this paper is simple and has no application conditions. In BP Neural Network handwritten digit recognition experiments, this method compared with the Bayesian regularization method improve the correct rate about 1.14-1.50 percentage points. Therefore, the method in this paper makes the algorithm more efficient.This method is validity.