Information Theory Coding信息论与编码（英文版） -梁建武 第2章 信息度量.ppt

Chapter 2.Basic Concepts of Info. Theory--Information statistical measure Introduction——preparation knowledge1.Information measure The information is measurable – which is the foundation to establish Info. Theory; Methods of information measure :Structure measure , statistical measure , semantic measure , fuzzy measure and so on; Statistical measure: this method counts with logarithm of the probability of the event to describe the uncertain thing and to obtain the info. content of the message, it also establishes a new concept --entropy ; Entropy is the most important concept in Shannon Info.Theory. 2.Single mark discrete source mathematical model Since the discrete source only involves a random event, it can be expressed by a discrete random variable. Single mark discrete mathematical model X，Y，Z are the random variables, refer to whole source ; is certain result of a random event or some element of the source. cannot confuse! 3Review 4.Mathematics knowledge Log(xy)=logx+logy Log(x/y)=logx-logy 2.1 Self –Info. And Conditional Self-Info. 2.1.1 Self-Info. Self-Info. Explanations： one will inevitably surprised when small probability event appears, therefore the information produced is rich; If the impossible event once to appear nearly, it will be an explosive news, amazing the world with a single brilliant feat. An event that most probably happened is during the people’s expectation. Even if it occurs, it doesn’t has any information . Especially when the inevitable event has occurred, it cannot give us any information . Nature of the self-Info. I(ai) I(ai) is a non- negative value； When P(ai) =1， I(ai)=0； When P(ai) =0， I(ai)= ∞ ； I(ai) is the monotone decreasing function of P(ai) Union self-Info. The source model (involves two random events) Union self-Info 2.1.2 Conditional Self-Info. negative value of logarithm of the conditional probability the information quantity which brought by the random event