第十一章 分类与预测.pptVIP

* SVM—Introduction Literature “Statistical Learning Theory” by Vapnik: extremely hard to understand, containing many errors too. C.?J.?C. Burges. A Tutorial on Support Vector Machines for Pattern Recognition. Knowledge Discovery and Data Mining, 2(2), 1998. Better than the Vapnik’s book, but still written too hard for introduction, and the examples are so not-intuitive The book “An Introduction to Support Vector Machines” by N. Cristianini and J. Shawe-Taylor Also written hard for introduction, but the explanation about the mercer’s theorem is better than above literatures The neural network book by Haykins Contains one nice chapter of SVM introduction * Lazy vs. Eager Learning Lazy vs. eager learning Lazy learning (e.g., instance-based learning): Simply stores training data (or only minor processing) and waits until it is given a test tuple Eager learning (the above discussed methods): Given a set of training set, constructs a classification model before receiving new (e.g., test) data to classify Lazy: less time in training but more time in predicting Accuracy Lazy method effectively uses a richer hypothesis space since it uses many local linear functions to form its implicit global approximation to the target function Eager: must commit to a single hypothesis that covers the entire instance space * Lazy Learner: Instance-Based Methods Instance-based learning: Store training examples and delay the processing (“lazy evaluation”) until a new instance must be classified Typical approaches k-nearest neighbor approach Instances represented as points in a Euclidean space. 近邻分类方法 近邻分类方法是基于实例的分类方法 不需要事先进行分类器的设计 直接使用训练集对未知类标号的数据样本进行分类 最近邻分类、k-近邻分类 * 最近邻法 最近邻法：nearest neighborhood classifier (nnc)，将与测试样本最近邻样本的类别作为决策的结果。 对一个C类别问题，每类有Ni个样本，i＝1，…，C，则第i类ωi的判别函数为： 决策规则： 最近邻法在原理上最直观，方法上也十分简单，明显的缺点就是计算量大，存储量大。 ‖·‖表示某种距离（相似性）度量，常用欧氏距离作为相似性度量。 * * The k-Nearest Neighbor Algorithm All instances correspond to points in the n-D space The nearest neighbor are defined in terms of