Exadata在1号店数据平台的应用实践.pptx

Exadata在1号店数据平台的应用实践 overview • as an Hardware with Intelligence – offloading – smart scan – storage index • why we used for BI OLAP – above all – ehcc Platform Storm • • • • • Maximum Availability Architecture High performance Scalable Exadata OLAP Distributed Database Systems Oracle cluster Nosql cluster Mysql cluster Hbase cluster 1号店数据平台整体架构 Service EDW real computesystem EXADATA Hadoop cluster Replication layer 流 技基 术于 实 时 应 用 北京IDC We need? Thousand ETL jobs PIS backend报表查询 自助取数平台 reports 个性精准化广 告业务 数据分析挖掘 CMS SCM • • • • 订单交易处理 财务、SCM报表 流量数据挖掘 日志数据分析 How to do? 流量明细基础 数据 日志数据 订单交易数据、 财务数据、 CRM数据 structured and unstructured Integration 1号店DW系统框架 Exadata Hadoop Mysql 构成:  Exadata：非流量数据的数据镜像层、数据整合层，数据中间层，应用层  Hadoop：流量数据的数据镜像层、数据整合层  Mysql：依据应用做数据集市 1号店Exadata Half Rack X4-2 migrate Quarter Rack X2-2  X2 active data guard  X4 one database many services  Why migrate Use Exadata Better——smart scan • The Bottleneck on Many (Most) Large Databases is between the Disk and the DB Server(s)! • How to Speed Up? – Make the Pipe Bigger/Faster – Reduce the Volume • 触发smart scan的前提(三条都满足): – Full scan: Full Table Scan or Fast Full Index Scan – Direct Path Reads: 并发读取数据到program global area(PGA) – 数据保存在exadata storage上: smart scan offloading是storage server独有 • Need Index or not? – When indexes are present, Exadata databases tend to over use them – Make them invisible first, and then remove them Use Exadata Better——EHCC • The hybrid columnar compression (HCC) Requirements: – Direct path loads – Impdp with DIRECT option or DIRECT path inserts – Alter table move • Data is stored in compression unit – not real columnar compression – advantage vs. disadvantage • If you update rows in HCC table, The entire compression unit is locked. The rows that are updated may be moved to a lower compression level such as No compression/ OLTP compression • How to use better Use Exadata Better——