A Learning Algorithm for Continually Running Fully Recurrent Neural Networks-英文文献.pdfVIP

A Learning Algorithm for Continually Running Fully Recurrent Neural Networks Ronald J Williams College of Computer Science Northeastern University Boston Massachusetts and David Zipser Institute for Cognitive Science University of California San Diego La Jolla California App ears in Neur al Computation pp Abstract The exact form of a gradientfollowing learning algorithm for completely recurrent net works running in continually sampled time is derived and used as the basis for practical algorithms for temp oral sup ervised learning tasks These algorithms have the advantage that they do not require a precisely dened training interval op erating while the network runs and the disadvantage that they require nonlo cal communication in the network b e ing trained and are computationally exp ensive These algorithms are shown to allow networks having recurrent connections to learn complex tasks requiring the retention of information over time p erio ds having either xed or indenite length Intro duction A ma jor problem in connectionist theory is to develop learning algorithms that can tap the full computational p ower of neural networks Much progress has b een made with feedforward net works and attention has recently turned to developing algorithms for networks with recurrent connections which have imp ortant capabilities not found in feedforward networks including at tractor dynamics and the ability to st