JournalofChineseLanguageandComputer11(2-中国语言文字网.doc

Journal of Chinese Language and Computer 11(2), Dec 2001, 136, Singapore Some Grammatical Considerations in the Determination of Unit in Word Segmentation Zhiwei FENG (Department of Electric Engineering and Computer Science, KAIST, Republic of Korea) Hock Kiat KOH (Nanyang Technological University, Singapore) Journal of Chinese Language and Computer 11(2), Dec 2001, 136, Singapore Some Grammatical Considerations in the Determination of Unit in Word Segmentation Zhiwei FENG (Department of Electric Engineering and Computer Science, KAIST, Republic of Korea) Hock Kiat KOH (Nanyang Technological University, Singapore) Abstract : In this paper, the author points out the weakness for the determination of segmentation unit in linguistic theory. This weakness comes from the inner contradiction between the basic linguistic conceptions on morpheme, word and phrase. In according to these conceptions, the free morpheme and singleton (simple word) become the intersection of morpheme and word. They are identical in logic but they belong to different categories in linguistics . morpheme word free morpheme = singleton bound morpheme compound It is the fundamental reason why so difficult to determine the unit of word segmentation. In Chinese language, the structural and functional similarity of compound and phrase brings about the situation to get worse and worse. In order to solve this problem, the author proposes the conception of NLP word. The NLP word is just the unit in word segmentation. Then the author systematically analyzes the grammatical factors to determine the NLP word. These factors are specified to following different approaches: Substitution test approach: to substitute the free morpheme in the checked- structure with other free morphem