bayes dicision贝叶斯决策的教程.pptVIP

* * * * * * * Discriminant function for discrete features So the decision surface is again a hyperplane. Optimality Consider a two-class case. Two ways to make a mistake in the classification: Misclassifying an observation from class 2 to class 1; Misclassifying an observation from class 1 to class 2. The feature space is partitioned into two regions by any classifier: R1 and R2 Optimality Optimality In the multi-class case, there are numerous ways to make mistakes. It is easier to calculate the probability of correct classification. Bayes classifier maximizes P(correct). Any other partitioning will yield higher probability of error. The result is not dependent on the form of the underlying distributions. * * * * * * * * * * * * * * * * * * * * * * * * * * * * Lecture 2.Bayesian Decision Theory Bayes Decision Rule Loss function Decision surface Multivariate normal and Discriminant Function Bayes Decision It is the decision making when all underlying probability distributions are known. It is optimal given the distributions are known. For two classes w1 and w2 , Prior probabilities for an unknown new observation: P(w1) : the new observation belongs to class 1 P(w2) : the new observation belongs to class 2 P(w1 ) + P(w2 ) = 1 It reflects our prior knowledge. It is our decision rule when no feature on the new object is available: Classify as class 1 if P(w1 ) P(w2 ) Bayes Decision We observe features on each object. P(x| w1) P(x| w2) : class-specific density The Bayes rule: Bayes Decision Likelihood of observing x given class label. Bayes Decision Posterior probabilities. Loss function Loss function: probability statement -- decision some classification mistakes can be more costly than others. The set of c classes: The set of possible actions: : deciding that an observation belongs to Loss when taking action i given the observation belongs to hidden class j: Loss function The expected loss: Given an observation with covari