数据挖掘课件84331.ppt

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * BUC: Partitioning Usually, entire data set can’t fit in main memory Sort distinct values, partition into blocks that fit Data Warehousing and OLAP Technology What is a data warehouse? A multi-dimensional data model Data warehouse architecture Data warehouse implementation Data warehouse Design OLAP Modeling Methods Optimization of Logical Model/Physical Model Plan and Implementation of Data Warehouse Further development of data cube technology From data warehousing to data mining 数据仓库的投资分析 数据仓库的应用目标 企业的核心业务 优化企业内部管理控制 为企业增加商业机会 建设数据仓库的必要性 企业内部复杂程度 高 低 低 高 必要性不强 优化企业内部 管理控制 必须建立： 核心业务， 增加商业机会 增加商业机会 客户数量 可通过计算ROI（Return of Investment）来衡量投资回报的价值 数据仓库主题的选择和阶段规划 数据仓库的实施是一个极为复杂的长期过程，因此，应选择当前最急需、能在短期内产生效益、业务模型清晰的任务首期实现 选择首期实现主题的参考原则： 优先实现管理者目前需求最迫切和最关心的主题 优先选择能在短期内产生效益的主题 推后选择业务逻辑准备不充分的主题 推后实施技术难度较大、可实现性较低、投资风险大的主题 维护阶段 数据仓库的维护极为重要，一般数据仓库在建立完成之后，都需要一至两年的维护 数据仓库的维护过程就是DSS逐步产生效益的过程 数据仓库后端工具 数据抽取(Data extraction): get data from multiple, heterogeneous, and external sources 数据清洗(Data cleaning): detect errors in the data and rectify them when possible 数据转换(Data transformation): convert data from legacy or host format to warehouse format 数据装载(Load): sort, summarize, consolidate, compute views, check integrity, and build indicies and partitions 刷新(Refresh): propagate the updates from the data sources to the warehouse Data Warehousing and OLAP Technology What is a data warehouse? A multi-dimensional data model Data warehouse architecture Data warehouse implementation Further development of data cube technology From data warehousing to data mining Discovery-Driven Exploration of Data Cubes Hypothesis-driven: exploration by user, huge search space Discovery-driven (Sarawagi et al.’98) pre-compute measures indicating exceptions, guide user in the data analysis, at all levels of aggregation Exception: signi