Divergence measures based on the Shannon entropy-英文文献.pdf

IEEE TRANSACTIONS ON INFORMATION THEORY. VOL. 37, NO. I , JANUARY 1991 145 Divergence Measures Based on the Shannon Entropy context. Rao’s objective was to obtain different measures of diversity [24] and the Jensen difference can be defined in terms Jianhua Lin, Member, IEEE of information measures other than the Shannon entropy func- tion. No specific detailed discussion was provided for the Jensen difference based on the Shannon entropy. Abstract -A new class of information-theoretic divergence measures based on the Shannon entropy is introduced. Unlike the well-known 11. THE KULLBACKI A N D J DIVERGENCEMEASURES Kullback divergences, the new measures do not require the condition of absolute continuity to be satisfied by the probability distributions in- Let X be a discrete random variable and let pI and p 2 be volved. More importantly, their close relationship with the variational two probability distributions of X . The I directed divergence distance and the probability of misclassification error are established in [17], [19] is defined as terms of bounds. These bounds are crucial in many applications of divergence measures. The new measures are also well characterized by the properties of nonnegativity, finiteness, semiboundedness, and boundedness. Index Terms-Divergence, dissimilarity measure,