Multivariate Linear Regression on Classifier Outputs a Capacity Study.pdfVIP

Multivariate Linear Regression on Classier Outputs a Capacity Study Yann Guermeur and Hel ene PaugamMoisy LIP URA CNRS ENS Lyon allee dItalie F Lyon Cedex France HelenePaugamMoisyYannGuermeurenslyon fr Patrick Gallinari LIP UMR CNRS Universite Paris Place Jussieu F Paris Cedex France PatrickGallinarilipfr Abstract We consider the problem of combining the outputs of several classiers trained indep endentl y to p erform a discrimination task in order to im prove the prediction accuracy of individ ual classiers We briey describ e the multivariate linear regression mo del which has already b een imple mented successfully for that purp ose and we study its capacity using generalizati ons of the notion of VC dimension Intro duction The idea of combining predictors instead of p erforming mo del selection has b een intensively investigated in statistics during the last three decades These studies have given birth to various ensemble methods characaterized either by the nature of the combiner or the training strategy However theoret ical evidence has b een mainly develop ed for regression whereas the sp ecicities of discrimination have seldom b een taken into account In we established how the Multivariate Linear Regression MLR mo del could b e implemented in order to improve the class p osterior probability estimates provided by a set of exp erts It was observed that this combiner compared favourably with the metho ds currently used on a central op en problem in predictive structural bi ology see