《An Overview of Statistical Learning Theory by Vladimir N. Vapnik》.pdf

988 IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 5, SEPTEMBER 1999 An Overview of Statistical Learning Theory Vladimir N. Vapnik Abstract— Statistical learning theory was introduced in the late B. Problem of Risk Minimization 1960’s. Until the 1990’s it was a purely theoretical analysis of the problem of function estimation from a given collection of data. In order to choose the best available approximation to the In the middle of the 1990’s new types of learning algorithms supervisor’s response, one measures the loss or discrepancy (called support vector machines) based on the developed theory between the response of the supervisor to were proposed. This made statistical learning theory not only a given input and the response provided by the a tool for the theoretical analysis but also a tool for creating learning machine. Consider the expected value of the loss, practical algorithms for estimating multidimensional functions. This article presents a very general overview of statistical learning given by the risk functional theory including both theoretical and algorithmic aspects of the theory. The goal of this overview is to demonstrate how the (2) abstract learning theory established conditions for generalization which are more general than those discussed in classical statis- The goal is to find the function which mini- tical paradigms and how the understanding of these conditions mizes the risk functional (over the class of functions inspired new algorithmic approa