Text Mining 网络文本挖掘[精品].ppt

Vector space is the play-yard * Classification Support Vector Machine * Latent Semantic Indexing Beyond Keyword matching Suppose I use the query “超女 冠军” to search， and there’s an article A about 李宇春 which contains no exact keyword of neither “超女”，nor “冠军”， how can we make it show up? * Latent Semantic Indexing 李宇春 广告 超女 冠军 代言 饮料 Doc1 1 1 1 1 0 0 Doc2 1 0 1 1 1 0 Doc3 1 1 0 0 1 1 Intuitively, we found 李宇春 is highly related to 超女，冠军 in most of documents Conceptually, we say李宇春 is closely related to, or even can be interpreted by超女，冠军. Mathematically, we say 李宇春 can be 超女，冠军 represented as a linear combination of超女，冠军 etc. * Latent Semantic Indexing Singular Vector Decomposition * Latent Semantic Indexing 李宇春 广告 超女 冠军 代言 Doc1 0.8 0.7 0.9 0.85 0 Doc2 1 0.2 1 1 1 Doc3 1 1 0.8 0.8 1 Now projected into a lower dimensional space * Clustering K-means * Hierarchical Clustering * Text Mining ABC 2Statistical Approach Na?ve Bayes * Text Mining ABC 2Statistical Approach Na?ve Bayes If we know how likely a document will look like given it’s class, can we know inversely the chance of the document to be any specific class, given the document itself. * Text Mining ABC 2Statistical Approach * Text Mining ABC 2Statistical Approach Hidden Markov Model * Hidden Markov Model If we observe the sequence x1x2x3, can we make a good guess on the sequence of status? This makes sense when we need to tag the words within a text to proper entities. Text Mining ABC 2Statistical Approach * Text Mining ABC 2Statistical Approach * What other players do with Text Mining? International IBM Microsoft Google These guys research on almost every thing!!! * AutonomyLeading solution provider on Unstructured Information Management Product IDOL? Server Retrieval Hyperlinking Summarization Taxonomy Generation Categorization Clustering Agents Profiling Collaboration Alerting * BuzzmetricsKey player in WOM monitoring Relevance Detection Classification Phrase Mining