A TUTORIAL ON PRINCIPAL COMPONENT （主成分的教程）.pdf

A TUTORIAL ON PRINCIPAL COMPONENT ANALYSIS Derivation, Discussion and Singular Value Decomposition Jon Shlens | jonshlens@ 25 March 2003 | Version 1 Principal component analysis (PCA) is a mainstay the assumptions behind this technique as well as pos- of modern data analysis - a black box that is widely sible extensions to overcome these limitations. used but poorly understood. The goal of this paper is The discussion and explanations in this paper are to dispel the magic behind this black box. This tutorial informal in the spirit of a tutorial. The goal of this focuses on building a solid intuition for how and why paper is to educate. Occasionally, rigorous mathe- principal component analysis works; furthermore, it matical proofs are necessary although relegated to crystallizes this knowledge by deriving from first prin- the Appendix. Although not as vital to the tutorial, cipals, the mathematics behind PCA . This tutorial the proofs are presented for the adventurous reader does not shy away from explaining the ideas infor- who desires a more complete understanding of the mally, nor does it shy away from the mathematics. math. The only assumption is that the reader has a The hope is that by addressing both aspects, readers working knowledge of linear algebra. Nothing more. of all levels will be able to gain a better understand- Please feel free to contact me with any suggestions, ing of the power of PCA as well as the when, the how corrections or comments. and the why of applying this technique. 2 Motivation: A Toy Example 1 Overview