Estimation of probabilities from sparse data for the language model component of a speech recognizer-英文文献.pdf

400 IEEE TRANSACTIONS ON ACOUSTICS, SPEECH,AND SIGNAL PROCESSING, VOL. ASP-35, NO. 3, MARCH 1987 Estimation of Probabilities from Sparse Data for the and the Turing’s estimate is Language Model Component of a Speech Recognizer SLAVA M. KATZ where Abstract-The description of a novel typeof rn-gram language model .*(x) = (.(x) + 1) n r o + l * is given. The modeloffers, via a nonlinear recursive procedure, a com- f k x ) putation and space efficient solutionto the problem of estimating prob- It follows from (1)-(3) that the total probability estimate, using abilities from sparse data. This solution compares favorably to other ( 5 ) , for the set of words (m-gram) that actually occurred in the proposed methods. While the method has been developed for and suc- sample is cessfully implemented in the IBM Real Time Speech Recognizers, its generality makes it applicable in other areas where the problem of es- c P T ( W ; ’ ) = 1 - nl- timating probabilities from sparse data arises. w;“:c ( w f ) 0 N This, in turn, leads to the estimate for the probability of observing Sparseness of data is an inherent property of any real text, and some previously unseen m-gram as a fraction nl/ N of “single-