Locality Preserving Projections.pdfVIP

Locality Preserving Projections Xiaofei He Partha Niyogi Department of Computer Science Department of Computer Science The University of Chicago The University of Chicago Chicago, IL 60637 Chicago, IL 60637 xiaofei@ niyogi@ Abstract Many problems in information processing involve some form of dimen- sionality reduction. In this paper, we introduce Locality Preserving Pro- jections (LPP). These are linear projective maps that arise by solving a variational problem that optimally preserves the neighborhood structure of the data set. LPP should be seen as an alternative to Principal Com- ponent Analysis (PCA) – a classical linear technique that projects the data along the directions of maximal variance. When the high dimen- sional data lies on a low dimensional manifold embedded in the ambient space, the Locality Preserving Projections are obtained by finding the optimal linear approximations to the eigenfunctions of the Laplace Bel- trami operator on the manifold. As a result, LPP shares many of the data representation properties of nonlinear techniques such as Laplacian Eigenmaps or Locally Linear Embedding. Yet LPP is linear and more crucially is defined everywhere in ambient space rather than just on the training data points. This is borne out by illustrative examples on some high dimensional data sets. 1. Introduction Suppose we have a collection of data points of -dimensional real vectors drawn from an unknown probability distribution. In increasingly many cases of interest in machine learn- ing and