Analysis of Unknown Lexical Items using Morphological and.pdf

Analysis of Unknown Lexical Items using Morphological and Syntactic Information with the TIMIT Corpus Scott M. Thede and Mary Harper Purdue University {thede, harper}@ecn, purdue, edu Abstract The importance of dealing with unknown words in Natural Language Processing (NLP) is growing as NLP systems are used in more and more applications. One aid in predicting the lexical class of words that do not appear in the lexicon (referred to as unknown words) is the use of syntactic parsing rules. The distinction between closed-class and open-class words together with morphological recognition appears to be pivotal in increasing the ability of the system to predict the lexical categories of unknown words. An experiment is performed to investigate the ability of a parser to parse unknown words using morphology and syntactic parsing rules without human intervention. This experiment shows that the performance of the parser is enhanced greatly when morphological recognition is used in conjunction with syntactic rules to parse sentences containing unknown words from the TIMIT corpus. 1 Introduction One of the problems facing natural language parsing (NLP) systems is the appearance of un- known words; words that appear in sentences, but are not contained within the lexicon for the system. This problem is one that will only get worse as NLP systems are used for more on-line computer applications. New words are continually added to the language, and people will often use words that a parsing system may not expect. This paper will empirically investigate how well a dictionary of closed-class words, syntactic parsing