A Probabilistic Approach to Compound Noun Indexing in Korean Texts.pdf

A Probabilistic Approach to Compound Noun Indexing in Korean Texts H y o u k R . P a r k a n d Y o u n g S. H a n a n d K a n g H . Lee K o r e a R D I n f o r m a t i o n C e n t e r / K I S T P . O . B o x 122 Y u S o n g T a e j o n , 305-600 , K o r e a { hrpark, yshan, khlee} @stissbs.kordic.re.kr K e y - S u n Choi C o m p u t e r S c i e n c e D e p a r t m e n t K A I S T Y u S o n g T a e j o n , 305-701 , K o r e a kschoi(O)world, kaist, ac. kr A b s t r a c t In this paper we address the prob- lem of compound noun indexing that is about segmenting or decomposing compound nouns into promising index terms. Compound nouns as index terms that usually subscribe to specific no- tions tend to increase the precision of retrieval performance. The use of the component nouns of a compound noun as index terms, on the other hand, may improve the recall performance, but can decrease the precision. Our proposed method to handle com- pound nouns with a goal to increase the recall while preserving the preci- sion computes the relevance of the com- ponent nouns of a compound noun to the document content by comparing the document sets that are supported by the component nouns and the terms of the document. The operational content of a term is represented as the proba- bilistic distribution of the term over the document set. Experiments with a set of 1,000 docu- ments show that our method gains 33% increase of retrieval performance com- pared to the indexing method without compound noun analysis, and is as good as manual decomposition by human ex- perts. 1 I n t r o d u c t i o n Automat ic indexing renders a form of document representation that visualizes the content of the document more explicitly. Indices that are care- fully chosen to represent a document will bring about the improvement of retrieval performance in accuracy and t ime efficiency. The potential of a candidate in