六、数据预处理---- 内容的提取与聚合data pre - web mining lab.pdf

六、数据预处理 内容的提取与聚合 Data Pre-processing in CCR 复旦大学2014年FIST课程 《计算新闻传播学》 Computational Communication Research Winson Peng 彭泰权 WKWSCI, NTU, Singapore Outline 1. What is data pre-processing? 2. Why data pre-processing is important in CCR? 3. How to pre-process data in CCR?  Readings  Homework 6 2 An Overview of Text as Data Methods Extracted from Grimmer Stewart, 2013 3 What is data pre-processing?  Make your data ready for statistical analysis and interpretation  Data can be in any formats , such as numbers, dates, time, text, video, audio, etc. o Our focus: Text (webpages/posts/blogs/tweets)  Why only text data? o New media research uses far more text data than numeric or multimedia data. o New media researchers are far less familiar with text mining than numeric data analysis. 4 Two Steps: Texts  Words Topics Topic • Tweets Extraction • Topics • Webpages • Words • Issues • Posts • Themes • …… • …… Word Ready for Extraction Analysis 5 Textual Information  Big amount of information is available in textual form in databases and online sources  Manual coding and effective extraction of useful information are not possible