Hadoop 介绍.pptx

Hadoop Development;Introduction to Hadoop and the Hadoop Ecosystem;HDFS;HDFS Basic Concepts;HDFS store;Name Node;Name Node;NameNode Availabilty;HA;Secondary Node;Options for Accessing HDFS;HDFS Command Example 1;HDFS Command Example 2;Importing Relational Data with Sqoop;Importing Relational Data with Sqoop;Import Entire DB with sqoop;Import Single Table with sqoop;Incremental Import 1 -- lastmodified ;Incremental Import 2 -- append;Import Partial Data Of Table With Sqoop;Free Query With Sqoop;Sqoop is perfect？;Export From HDFS To DB;Hive And Impala;Why Use Hive and Impala;HiveQL;Hive Quries;Hive Quries;Impala Shell;Modeling and Managing Data with Impala and Hive;How Hive and Impala Load and Store Data;How Hive and Impala Load and Store Data;Create Table With Basic Syntax ;Create Table Based On Existing Schema;Create Table Based On Existing Schema;External Table;Exploring Table;Exploring Table;Exploring Table;Use Hue MetaStore Tables;Load Data Into Tables;Load Data Into Tables;Overwriting Data From Files;Appending Selected Records to a Table;From Sqoop;Data Partitioning;Data Partitioning;Data Partitioning;Data Partitioning;Data Partitioning;Data Partitioning;Data Partitioning;Data Partitioning -- Static partitioning;Data Partitioning -- Static partitioning;NO Data Partitioning;Hive;Impala; 1、少写磁盘 与 MapReduce相比：Impala把整个查询分成一执行计划树，而不是一连串的MapReduce任务，在分发执行计划后，Impala使用拉式获取 数据的方式获取结果，把结果数据组成按执行树流式传递汇集，减少的了把中间结果写入磁盘的步骤，再从磁盘读取数据的开销。 2、Impala使用服务的方式避免 每次执行查询都需要启动的开销，即相比Hive没了MapReduce启动时间。 3、使用LLVM产生运行代码，针对特定查询生成特定代码，同时使用Inline的方式减少函数调用的开销，加快执行效率。 - LLVM是构架编译器(compiler)的框架系统，以C++编写而成 4、充分利用可用的硬件指令（SSE4.2）。;Impala Weakness;Capturing Data With Flume;What Is Flume;Flume Data Flow;Notable Built-in Flume Sources;Built-in Flume Sinks;Built-In Flume Channels;Flume Agent Configuration File;Starting a Flume Agent