mse best learning curves selected from the four parts 资讯工程所 .pptVIP

Chapter 4Multilayer Perceptrons Introduction Multilayer perceptrons The network is consists of a set of sensory units, that constitute the input layer, one or more hidden layers of computation nodes, and an output layer of computation nodes. 屬於supervised learning，是一種基於error-correction learning的error back-propagation algorithm 包含兩個階段：forward pass和backward pass Forward pass: an activity pattern is applied to the sensory nodes , and its effect propagates through the network layer by layer. Backward pass: the synaptic weights are all adjusted in accordance with an error-correction rule. The error signal is propagated backward through the network. Introduction (cont.) A multilayer perceptron has three distinctive characteristic: The mode of each neuron in the network includes a nonlinear activation function. Sigmoidal nonlinearity The network contains one or more layers of hidden neurons. Enable the network to learn complex tasks by extracting progressively more meaningful features from the input patterns. The network exhibits a high degrees of connectivity, determined by the synapses of the network. Some preliminaries The architectural graph of a MLP with two hidden layers and an output layer. Some preliminaries (cont.) Two kinds of signals are identified: Function signals Comes in at the input end of the network, propagates forward through the network, and emerges at the output end of the network as an output signal. Error signal Originates at an output neuron of the network, and propagates backward through the network Some preliminaries (cont.) 在multilayer perceptron中的每個隱藏層或輸出層的神經元都被設計來執行兩種計算功能： 神經元的輸出是輸入訊號經由權重的連結經一非線性轉換所得到的結果 用來估計梯度向量(gradient vector) ，以回傳到整個網路 Signal-flow graph highlighting the details of output neuron j Back-propagation algorithm (cont.) The error signal at the output of neuron j at iteration n is defined by The instantaneous value of the total error energy is defined as The average squared error energy is obtained by summing E(n) over a