web_挖掘.ppt

Data Mining: Concepts and Techniques Web Mining The WWW is huge, widely distributed, global information service center for Information services: news, advertisements, consumer information, financial management, education, government, e-commerce, etc. Hyper-link information Access and usage information WWW provides rich sources for data mining Challenges Too huge for effective data warehousing and data mining Too complex and heterogeneous: no standards and structure Mining the World-Wide Web Growing and changing very rapidly Broad diversity of user communities Only a small portion of the information on the Web is truly relevant or useful 99% of the Web information is useless to 99% of Web users How can we find high-quality Web pages on a specified topic? Web search engines Index-based: search the Web, index Web pages, and build and store huge keyword-based indices Help locate sets of Web pages containing certain keywords Deficiencies A topic of any breadth may easily contain hundreds of thousands of documents Many documents that are highly relevant to a topic may not contain keywords defining them (polysemy) Web Mining: A more challenging task Searches for Web access patterns Web structures Regularity and dynamics of Web contents Problems The “abundance” problem Limited coverage of the Web: hidden Web sources, majority of data in DBMS Limited query interface based on keyword-oriented search Limited customization to individual users Web Mining Taxonomy Mining the World-Wide Web Mining the World-Wide Web Mining the World-Wide Web Mining the World-Wide Web Mining the World-Wide Web Mining the Webs Link Structures Finding authoritative Web pages Retrieving pages that are not only relevant, but also of high quality, or authoritative on the topic Hyperlinks can infer the notion of authority The Web consists not only of pages, but also of hyperlinks pointing from one page to another These hyperlinks contain an enormous amount of latent human annotation A hyperli