文章基本信息

标题：Improved Generalization in Recurrent Neural Networks Using the Tangent Plane Algorithm
本地全文：下载
作者：P May ; E Zhou ; C. W. Lee 等
期刊名称：International Journal of Advanced Computer Science and Applications(IJACSA)
印刷版ISSN：2158-107X
电子版ISSN：2156-5570
出版年度：2014
卷号：5
期号：3
DOI：10.14569/IJACSA.2014.050317
出版社：Science and Information Society (SAI)
摘要：The tangent plane algorithm for real time recurrent learning (TPA-RTRL) is an effective online training method for fully recurrent neural networks. TPA-RTRL uses the method of approaching tangent planes to accelerate the learning processes. Compared to the original gradient descent real time recurrent learning algorithm (GD-RTRL) it is very fast and avoids problems like local minima of the search space. However, the TPA-RTRL algorithm actively encourages the formation of large weight values that can be harmful to generalization. This paper presents a new TPA-RTRL variant that encourages small weight values to decay to zero by using a weight elimination procedure built into the geometry of the algorithm. Experimental results show that the new algorithm gives good generalization over a range of network sizes whilst retaining the fast convergence speed of the TPA-RTRL algorithm.
关键词：thesai; IJACSA; thesai.org; journal; IJACSA papers; real time recurrent learning; tangent plane; generalization; weight elimination; temporal pattern recognition; non-linear process control