《DHSch3part2.ppt

Chapter 3:Maximum-Likelihood and Bayesian Parameter Estimation (part 2) Bayesian Estimation (BE) Bayesian Parameter Estimation: Gaussian Case Bayesian Parameter Estimation: General Estimation Problems of Dimensionality Computational Complexity Component Analysis and Discriminants Hidden Markov Models Bayesian Estimation (Bayesian learning to pattern classification problems) In MLE ? was supposed fix In BE ? is a random variable The computation of posterior probabilities P(?i | x) lies at the heart of Bayesian classification Goal: compute P(?i | x, D) Given the sample D, Bayes formula can be written To demonstrate the preceding equation, use: Bayesian Parameter Estimation: Gaussian Case Goal: Estimate ? using the a-posteriori density P(? | D) The univariate case: P(? | D) ? is the only unknown parameter (?0 and ?0 are known!) Reproducing density Identifying (1) and (2) yields: The univariate case P(x | D) P(? | D) computed P(x | D) remains to be computed! It provides: (Desired class-conditional density P(x | Dj, ?j)) Therefore: P(x | Dj, ?j) together with P(?j) And using Bayes formula, we obtain the Bayesian classification rule: Bayesian Parameter Estimation: General Theory P(x | D) computation can be applied to any situation in which the unknown density can be parametrized: the basic assumptions are: The form of P(x | ?) is assumed known, but the value of ? is not known exactly Our knowledge about ? is assumed to be contained in a known prior density P(?) The rest of our knowledge ? is contained in a set D of n random variables x1, x2, …, xn that follows P(x) The basic problem is: “Compute the posterior density P(? | D)” then “Derive P(x | D)” Using Bayes formula, we have: And by independence assumption: Problems of Dimensionality Problems involving 50 or 100 features (binary valued) Classification accuracy depends upon the dimensionality and the amount of training data Case of two classes multivariate normal with t