A Two-Stage Approach to Domain Adaptation for Statistical 统计域的适应一二个阶段的方法.ppt

Nov 7, 2007 CIKM2007 A Two-Stage Approach to Domain Adaptation for Statistical Classifiers Jing Jiang ChengXiang Zhai Department of Computer Science University of Illinois at Urbana-Champaign What is domain adaptation? Example: named entity recognition Example: named entity recognition Domain difference ? performance drop Another NER example Other examples Spam filtering: Public email collection ? personal inboxes Sentiment analysis of product reviews Digital cameras ? cell phones Movies ? books Can we do better than standard supervised learning? Domain adaptation: to design learning methods that are aware of the training and test domain difference. How do we solve the problem in general? Observation 1 Observation 1 Observation 2 Observation 2 General idea: two-stage approach Goal Regular classification Generalization: to emphasize generalizable features in the trained model Adaptation: to pick up domain-specific features for the target domain Regular semi-supervised learning Comparison with related work We explicitly model generalizable features. Previous work models it implicitly [Blitzer et al. 2006, Ben-David et al. 2007, Daumé III 2007]. We do not need labeled target data but we need multiple source (training) domains. Some work requires labeled target data [Daumé III 2007]. We have a 2nd stage of adaptation, which uses semi-supervised learning. Previous work does not incorporate semi-supervised learning [Blitzer et al. 2006, Ben-David et al. 2007, Daumé III 2007]. Implementation of the two-stage approach with logistic regression classifiers Logistic regression classifiers Learning a logistic regression classifier Generalizable features in weight vectors We want to decompose w in this way Feature selection matrix A Decomposition of w Decomposition of w Decomposition of w Framework for generalization Framework for adaptation How to find A? (1) Joint optimization Alternating optimization How to find A? (2) Domain cross validation Idea: training on (K – 1