包邮HADOOP专家:管理.调优与SPARKYARNHDFS安全

目录
第Ⅰ部分　Hadoop架构与Hadoop集群介绍
第1章　Hadoop与Hadoop环境介绍............................................................................... 3
Hadoop简介.........................................................................................................................4
Hadoop 的特性............................................................................................................5
Hadoop 与大数据........................................................................................................5
Hadoop 的典型应用场景............................................................................................6
传统数据库系统..........................................................................................................7
数据湖..........................................................................................................................9
大数据、数据科学和Hadoop ..................................................................................10
Hadoop集群与集群计算................................................................................................... 11
集群计算.................................................................................................................... 11
Hadoop 集群..............................................................................................................12
Hadoop组件和Hadoop生态..............................................................................................14
Hadoop管理员需要做些什么...........................................................................................16
Hadoop 管理—新的范式......................................................................................17
关于Hadoop 管理你需要知道的.............................................................................18
Hadoop 管理员的工具集..........................................................................................19
Hadoop 1和Hadoop 2的关键区别....................................................................................19
架构区别....................................................................................................................20
高可用性....................................................................................................................20
多计算引擎................................................................................................................21
xiv 目录
分离处理和调度........................................................................................................21
Hadoop 1 和Hadoop 2 中的资源分配.....................................................................22
分布式数据处理：MapReduce和Spark、Hive、Pig ......................................................22
MapReduce ................................................................................................................22
Apache Spark .............................................................................................................23
Apache Hive ...............................................................................................................24
Apache Pig .................................................................................................................24
数据整合：Apache Sqoop、Apache Flume和Apache Kafka ..........................................25
Hadoop管理中的关键领域...............................................................................................26
集群存储管理............................................................................................................26
集群资源分配............................................................................................................26
作业调度....................................................................................................................27
Hadoop 数据安全......................................................................................................27
总结....................................................................................................................................28
第2章　Hadoop架构介绍............................................................................................. 31
Hadoop与分布式计算.......................................................................................................31
Hadoop 架构......................................................................................................................32
Hadoop 集群..............................................................................................................33
主节点和工作节点....................................................................................................33
Hadoop 服务..............................................................................................................34
数据存储—Hadoop分布式文件系统...........................................................................35
HDFS 特性................................................................................................................35
HDFS 架构................................................................................................................36
HDFS 文件系统........................................................................................................38
NameNode 操作........................................................................................................41
利用YARN（Hadoop操作系统）进行数据处理...........................................................45
YARN 的架构............................................................................................................46
ApplicationMaster 如何与ResourceManager 协作进行资源分配..........................51
总结....................................................................................................................................54
第3章　创建和配置一个简单的Hadoop集群................................................................ 55
Hadoop发行版本和安装类型...........................................................................................56
目录xv
Hadoop 发行版本......................................................................................................56
Hadoop 安装类型......................................................................................................57
设置一个伪分布式Hadoop集群.......................................................................................58
满足操作系统的要求................................................................................................58
修改内核参数............................................................................................................59
设置SSH ...................................................................................................................64
Java 需求....................................................................................................................65
安装Hadoop ..............................................................................................................66
创建必要的Hadoop 用户.........................................................................................66
创建必要的目录........................................................................................................67
Hadoop初始配置...............................................................................................................67
环境变量配置文件....................................................................................................69
只读默认配置文件....................................................................................................70
site 专用配置文件..............................................................