Incorporating non-local information into information extraction systems by gibbs sampling-英文文献.pdf

Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling Jenny Rose Finkel, Trond Grenager, and Christopher Manning Computer Science Department Stanford University Stanford, CA 94305 {jrfinkel, grenager, mannning }@ Abstract of it in natural language processing.1 Here, we use it to add non-local dependencies to sequence models Most current statistical natural language process- for information extraction. ing models use only local features so as to permit Statistical hidden state sequence models, such dynamic programming in inference, but this makes them unable to fully account for the long distance as Hidden Markov Models (HMMs) (Leek, 1997; structure that is prevalent in language use. We Freitag and McCallum, 1999), Conditional Markov show how to solve this dilemma with Gibbs sam- Models (CMMs) (Borthwick, 1999), and Condi- pling , a simple Monte Carlo method used to per- form approximate inference in factored probabilis- tional Random Fields (CRFs) (Lafferty et al., 2001) tic models. By using simulated annealing in place are a prominent recent approach to information ex- of Viterbi decoding in sequence models such as traction tasks. These models all encode the Markov HMMs, CMMs, and CRFs, it is possible to incorpo- rate non-local structure while preserving tractable property: decisions about the state at a particular po-