feature selection.ppt

Data Mining: Concepts and Techniques Feature Selection Methods Qiang Yang MSC IT 521 The space of choices is large Feature Selection using Chi-Square Question: Are attributes A1 and A2 independent? If they are very dependent, we can remove eitherA1 or A2 If A1 is independent on a class attribute A2, we can remove A1 from our training data Chi-Squared Test (cont.) The Weather example: Observed Count The Weather example: Expected Count Question: How different between observed and expected? Chi-Squared Table: what does it mean? If calculated value is much greater than in the table, then you have reason to reject the independence assumption When your calculated chi-square value is greater than the chi2 value shown in the 0.05 column (3.84) of this table ? you are 95% certain that attributes are actually dependent! i.e. there is only a 5% probability that your calculated X2 value would occur by chance Principal Component Analysis主成分分析: one attribute first Question: how much spread is in the data along the axis? (distance to the mean) Variance=Standard deviation^2 Now consider two dimensions More than two attributes: covariance matrix Contains covariance values between all possible dimensions (=attributes): Example for three attributes (x,y,z): Background: eigenvalues AND eigenvectors Eigenvectors（ 特征向量）e : C e =? e How to calculate e and ?: Calculate det(C-?I), yields a polynomial (degree n) Determine roots to det(C-?I)=0, roots are eigenvalues ? Check out any math book such as Elementary Linear Algebra by Howard Anton, Publisher John,Wiley Sons Or any math packages such as MATLAB An Example Covariance Matrix C= Using MATLAB, we find out: Eigenvectors: e1=(-0.98, 0.21), ?1=51.8 e2=(0.21, 0.98), ?2=560.2 Thus the second eigenvector is more important! If we only keep one dimension: e2 We keep the dimension of e2=(0.21, 0.98) We can obtain the final data as Summary of PCA PCA is used for reducing the number of numerical attributes The key is in data tra