模式分类英文课件prch5partding.pptVIP

Pattern ClassificationAll materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley Sons, 2000 with the permission of the authors and the publisher 5.4 The Two-Category Linearly Separable Case Linearly separable n samples belong to , if there exits a linear discriminant function that classifies all of them correctly, the samples are said to be linearly separable. Weight vector a is called a separating vector or solution vector Normalization Replacing all samples labeled by their negatives is called normalization with this normalization we only need to look for a weight vector a such that Solution Region Margin the distance between old boundaries and new boundaries is The problem of finding a linear discriminant function will be formulated as a problem of minimizing a criterion function Gradient Descent Procedures define a criterion function that is minimized if a is a solution vector. This can often be solved by a gradient descent procedure. Choice of learning rate Note: from (12) we have Substituting this into Eq.13. and minimize J(a(k+1)), we can get Eq.(14) Newton Descent Note: Newton’s algorithm vs the simple gradient decent algorithm 5.5 Minimizing The Perception Criterion Function The Perception Criterion Function Batch Perception Algorithm Comparison of Four Criterion functions Perceptron Criterion as function of weights Single-Sample Correction Interpretation of Single-Sample Correction Some Direct Generalizations Variable-Increment Perceptron with Margin Algorithm Convergence Batch Variable Increment Perception How to Choose b Balanced Winnow Algorithm 5.6 Relaxation Procedures Decent Algorithm criterion function: two problems of this criterion function criterion function: Update rule : Batch Relaxation with Margin Single-Sample Relaxation with margin