基于数据挖掘的微博舆情监测与分析系统研究与实现-软件工程专业论文.docx

AbstractAbstract Abstract Abstract Recently,with the rapid development of Intemet，It makes people to face the immensity data at loose ends．And mean times，network security issues become increasingly prominent，Emergencies occur frequently result in tremendous social loss， therefore more and more attentions have been paid by people．Microblog is a new kind of Intemet media platform that appeared in several years ago，Microblog is becoming increasingly important in our life．But on the other hand，there are some disharmony and uncivilized behavior,and even some expression of anti-government and disrupting society．For the above situation，Microblog public opinion waming and detecting technology came into being．The public opinion detecting and analysis system to be a solution that to correctly guide public opinion，clean network environment and provide a new management means for relevant government departments．This system main goal is provide a service that public opinion detecting and analysis for relevant government departments to guide public opinion by crawl the Microblog web to obtain web Information，save the data and then display the results after data analysis． The system is mainly consisted of the following three parts： The first part is information collection．Through the studies in the Open Source Framework Nutch，Crawler and Web Collector,this system designs a multithreaded web crawler for Microblog to crawl the microblog web information and save it combined with the best of them． The second part is data analysis．This paper contributes a method，which aims at identify hot topics in Microblog based on k-means．In this method，after the pre‘-disposition of the Microblog data,by dividing time·-window,by extracting topic words according to the two factors of increasing rate of word frequency and relative word frequency from Microblog data in every time-window,clustering the topic words according to the similarity among them，then sieving for a suitable cluster of 万方数据 Abstract