`
Kevin12
  • 浏览: 234920 次
  • 性别: Icon_minigender_1
  • 来自: 上海
社区版块
存档分类
最新评论

hadoop2.6.0集群的搭建方法

hadoopubuntuvmware 
阅读更多
1.集群环境的安装
1.1工具软件版本说明（软件尽量去官网下载）：
VMware Workstation :10.0.0 build-1295980
Ubuntn:Ubuntn15.10
JDK：jdk-8u60-linux-x64.tar.gz
Hadoop:hadoop-2.6.0.tar.gz
SecureCRT：Version 6.2.3 (build 313)
WinSCP:5.1.6(build 3394)
VM先按照ubuntu虚拟机步骤网上搜下，自行安装。
Ubuntu15.10设置用root账号登录:http://kevin12.iteye.com/blog/2271687
VMware中安装了ubuntu全屏设置: http://kevin12.iteye.com/blog/2271690
1.2 虚拟机的一般配置
1.2.1修改主机名称：运行命令vim /etc/hostname 将原先的主机名称改成master1,保存并退出，然后重启虚拟机即可；可以用命令hostname验证一下；
1.2.2 设置静态ip：可参考：http://kevin12.iteye.com/blog/2273491
1.2.3 绑定主机名和IP
运行vim /etc/hosts 绑定后保存并退出，如下图：

1.2.4 关闭防火墙
ubuntu防火墙用的是ufw
查看防火墙状态：sudo ufw status
关闭防火墙：sudo ufw disable
1.2.5 设置免密码登陆
Ubuntu默认安装openssh-client，但是木有安装server，如果运行ssh localhost会报错误：ssh: connect to host localhost port 22: Connection refused
通过命令sudo apt-get install ssh进行安装，过程如下： 
root@master1:~# sudo apt-get install ssh
正在读取软件包列表... 完成
正在分析软件包的依赖关系树       
正在读取状态信息... 完成       
将会安装下列额外的软件包：
  libck-connector0 ncurses-term openssh-client openssh-server
  openssh-sftp-server ssh-import-id
建议安装的软件包：
  libpam-ssh keychain monkeysphere rssh molly-guard
下列【新】软件包将被安装：
  libck-connector0 ncurses-term openssh-server openssh-sftp-server ssh
  ssh-import-id
下列软件包将被升级：
  openssh-client
升级了 1 个软件包，新安装了 6 个软件包，要卸载 0 个软件包，有 106 个软件包未被升级。
需要下载 8,784 B/1,262 kB 的软件包。
解压缩后会消耗掉 3,523 kB 的额外空间。
您希望继续执行吗？ [Y/n] Y
【警告】：下列软件包不能通过验证！
  libck-connector0 openssh-client openssh-sftp-server openssh-server ssh
  ncurses-term ssh-import-id
不经验证就安装这些软件包吗？ [y/N] y
获取：1 http://cn.archive.ubuntu.com/ubuntu/ wily/main libck-connector0 amd64 0.4.6-5 [8,784 B]
下载 8,784 B，耗时 0秒 (127 kB/s)     
正在预设定软件包 ...
正在选中未选择的软件包 libck-connector0:amd64。
(正在读取数据库 ... 系统当前共安装有 180732 个文件和目录。)
正准备解包 .../libck-connector0_0.4.6-5_amd64.deb  ...
正在解包 libck-connector0:amd64 (0.4.6-5) ...
正准备解包 .../openssh-client_1%3a6.9p1-2ubuntu0.1_amd64.deb  ...
正在将 openssh-client (1:6.9p1-2ubuntu0.1) 解包到 (1:6.9p1-2) 上 ...
正在选中未选择的软件包 openssh-sftp-server。
正准备解包 .../openssh-sftp-server_1%3a6.9p1-2ubuntu0.1_amd64.deb  ...
正在解包 openssh-sftp-server (1:6.9p1-2ubuntu0.1) ...
正在选中未选择的软件包 openssh-server。
正准备解包 .../openssh-server_1%3a6.9p1-2ubuntu0.1_amd64.deb  ...
正在解包 openssh-server (1:6.9p1-2ubuntu0.1) ...
正在选中未选择的软件包 ssh。
正准备解包 .../ssh_1%3a6.9p1-2ubuntu0.1_all.deb  ...
正在解包 ssh (1:6.9p1-2ubuntu0.1) ...
正在选中未选择的软件包 ncurses-term。
正准备解包 .../ncurses-term_5.9+20150516-2ubuntu1_all.deb  ...
正在解包 ncurses-term (5.9+20150516-2ubuntu1) ...
正在选中未选择的软件包 ssh-import-id。
正准备解包 .../ssh-import-id_4.5-0ubuntu1_all.deb  ...
正在解包 ssh-import-id (4.5-0ubuntu1) ...
正在处理用于 man-db (2.7.4-1) 的触发器 ...
正在处理用于 ureadahead (0.100.0-19) 的触发器 ...
正在处理用于 systemd (225-1ubuntu9) 的触发器 ...
正在处理用于 ufw (0.34-2) 的触发器 ...
正在设置 libck-connector0:amd64 (0.4.6-5) ...
正在设置 openssh-client (1:6.9p1-2ubuntu0.1) ...
正在设置 openssh-sftp-server (1:6.9p1-2ubuntu0.1) ...
正在设置 openssh-server (1:6.9p1-2ubuntu0.1) ...
Creating SSH2 RSA key; this may take some time ...
2048 SHA256:Uo5idLAsqmt7e8rAUxXt4hkB0L33zzVgIKv6Gx/TnX4 root@master1 (RSA)
Creating SSH2 DSA key; this may take some time ...
1024 SHA256:fFT1QElhDnmT//1tFoymVPKUVN/b/EnL+ODdzooF7no root@master1 (DSA)
Creating SSH2 ECDSA key; this may take some time ...
256 SHA256:ctbDQwKOmzr0Ixl43vm1akZAP360sbX4rHWk5FZ2gpo root@master1 (ECDSA)
Creating SSH2 ED25519 key; this may take some time ...
256 SHA256:NP0yHA9iwPNNXN3jXz6h+V+J/lYgGXJd5f2MUV4XG+8 root@master1 (ED25519)
正在设置 ncurses-term (5.9+20150516-2ubuntu1) ...
正在设置 ssh-import-id (4.5-0ubuntu1) ...
正在处理用于 ufw (0.34-2) 的触发器 ...
正在设置 ssh (1:6.9p1-2ubuntu0.1) ...
正在处理用于 libc-bin (2.21-0ubuntu4) 的触发器 ...
正在处理用于 ureadahead (0.100.0-19) 的触发器 ...
正在处理用于 systemd (225-1ubuntu9) 的触发器 ...
root@master1:~# /etc/init.d/ssh start
[ ok ] Starting ssh (via systemctl): ssh.service.
root@master1:~# ps -e | grep ssh
 3146 ?        00:00:00 sshd
//在home目录下面，执行命令ssh -keygen -t rsa  并一路回车即可；再运行命令cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
root@master1:~# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:CCBcvdZn1zlL033Gv6rdQqC1zY0fM73emg+ra+ELDt0 root@master1
The key's randomart image is:
+---[RSA 2048]----+
|o o..            |
| o . .           |
|    . o     . o..|
|     + o o + * .=|
|    . . S + * *.+|
|         o o.* =o|
|        . o.E.o *|
|         o .+..B.|
|          .o=*B=o|
+----[SHA256]-----+

root@master1:~# ssh localhost
The authenticity of host 'localhost (127.0.0.1)' can't be established.
ECDSA key fingerprint is SHA256:ctbDQwKOmzr0Ixl43vm1akZAP360sbX4rHWk5FZ2gpo.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 15.10 (GNU/Linux 4.2.0-16-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

107 packages can be updated.
0 updates are security updates.


The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

root@master1:~#ssh localhost  --再次运行时就不用输密码了
Welcome to Ubuntu 15.10 (GNU/Linux 4.2.0-16-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

107 packages can be updated.
0 updates are security updates.

Last login: Tue Jan 19 22:17:00 2016 from 127.0.0.1
root@master1:~#

2.安装jdk
2.1 将jdk-8u60-linux-x64.tar.gz拷贝到虚拟机中，用命令tar -zxvf jdk-8u60-linux-x64.tar.gz 解压，并将解压包移动到/usr/local/jdk下2.安装JDK
2.2 编辑.bashrc文件配置java的环境变量，添加下面的代码,保存并退出，执行命令：source /.bashrc使配置生效。配置如下： 
export JAVA_HOME=/usr/local/jdk/jdk1.8.0_60
export JRE_HOME=${JAVA_HOME}/jre
export PATH=.:${JAVA_HOME}/bin:$PATH
export CLASS_PATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=.:${JAVA_HOME}/bin:$PATH

2.3 验证命令:java -version,出现下面的代码说明jdk安装成功； 
root@jylu:~# java -version
java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode

3.将master1克隆出来四台节点，分别命名为worker1,worker2,worker3，worker4
3.1 修改worker1的主机名 vim /etc/hostname
3.2 设置worker1的静态ip
3.3 设置worker1的静态ip映射  192.168.112.131 worker1
3.4 关闭防火墙  然后重启
3.5 设置ssh：删除.ssh目录下的内容，重新生成ssh
3.6 将worker1中~/.ssh/id_rsa.pub的内容拷贝到master1中的authorized_keys中
按照上面同样的方法操作worker2,worker3,最后修改将master1中的authorized_keys复制到三台worker中的~/.ssh目录下；
修改master1的/etc/hosts文件，worker的机器ip映射添加进来，如下： 
192.168.112.130 master1
192.168.112.131 worker1
192.168.112.132 worker2
192.168.112.133 worker3
192.168.112.134 worker4

用scp命令将master1中的authorized_keys拷贝到其他4个节点上，命令如下： 
scp ~/.ssh/authorized_keys root@worker1:~/.ssh/
scp ~/.ssh/authorized_keys root@worker2:~/.ssh/
scp ~/.ssh/authorized_keys root@worker3:~/.ssh/
scp ~/.ssh/authorized_keys root@worker4:~/.ssh/

用ssh worker1 第一次要输入密码，之后就不用输入密码了，其他的节点也是如此。 
4.在master1上安装hadoop

4.1官网下载hadoop-2.6.0.tar.gz，复制到ubuntu虚拟机/usr/local/中，运行命令tar -zxvf hadoop-2.6.0.tar.gz ./hadoop 命令解压。
配置相关文件：
配置hadoop的环境变量   ~/.bashrc 
export JAVA_HOME=/usr/local/jdk/jdk1.8.0_60
export JRE_HOME=${JAVA_HOME}/jre
export CLASS_PATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export SCALA_HOME=/usr/local/scala/scala-2.10.4
export HADOOP_HOME=/usr/local/hadoop/hadoop-2.6.0
export HADOOP_CONF_DIR=${HADOOP_HOME}/etc/hadoop
export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_HOME}/lib/native
export HADOOP_OPTS="-Djava.library.path=${HADOOP_HOME}/lib"
export PATH=.:${JAVA_HOME}/bin:${SCALA_HOME}/bin:${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin:$PATH


说明：
HADOOP_CONF_DIR配置是为了更好的支持yarn的运行，包括spark运行yarn;
下面两项可选
export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_HOME}/lib/native
export HADOOP_OPTS="-Djava.library.path=${HADOOP_HOME}/lib"

core-site.xml配置： 
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://master1:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/usr/local/hadoop/hadoop-2.6.0/tmp</value>
</property>
<property>
<name>hadoop.native.lib</name>
<value>true</value>
<description>Should native hadoop libraries,if present,be used.</description>
</property>
</configuration>

hdfs-site.xml配置： 
<configuration>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>master1:50090</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>/usr/local/hadoop/hadoop-2.6.0/dfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>/usr/local/hadoop/hadoop-2.6.0/dfs/data</value>
</property>
<property>
<name>dfs.namenode.checkpoint.dir</name>
<value>file:///usr/local/hadoop/hadoop-2.6.0/dfs/namesecondary</value>
</property>
</configuration>

yarn-site.xml配置： 
<configuration>
<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.resourcemanager.hostname</name>
<value>master1</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>

yarn-env.sh配置： 
export JAVA_HOME=/usr/local/jdk/jdk1.8.0_60

hadoop-env.sh配置： 
export JAVA_HOME=/usr/local/jdk/jdk1.8.0_60
export HADOOP_HOME=/usr/local/hadoop/hadoop-2.6.0
export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_HOME}/lib/native
export HADOOP_OPS="-Djava.library.path=${HADOOP_HOME}/lib"

slaves配置： 
worker1
worker2
worker3
worker4

4.2在master1上配置好后，用scp命令将配置好的环境变量.bashrc和hadoop拷贝到其他四台worker节点上。
拷贝.bashrc命令： 
scp ~/.bashrc root@worker1:~/
scp ~/.bashrc root@worker2:~/
scp ~/.bashrc root@worker3:~/
scp ~/.bashrc root@worker4:~/

然后每台worker节点上运行命令source ~/.bashrc，使之生效。
拷贝配置好的hadoop命令： 
scp -r /usr/local/hadoop/ root@worker1:/usr/local/
scp -r /usr/local/hadoop/ root@worker2:/usr/local/
scp -r /usr/local/hadoop/ root@worker3:/usr/local/
scp -r /usr/local/hadoop/ root@worker4:/usr/local/

查看worker上的/usr/local/目录下面是否拷贝过来。
4.3 在master1上格式化namenode 
root@master1:/usr/local/hadoop/hadoop-2.6.0/bin# hdfs namenode -format
16/01/23 21:57:57 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = master1/192.168.112.130
STARTUP_MSG:   args = [-format]
STARTUP_MSG:   version = 2.6.0
STARTUP_MSG:   classpath = /usr/local/hadoop/hadoop-2.6.0/etc/hadoop:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hamcrest-core-1.3.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/curator-client-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-cli-1.2.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jetty-util-6.1.26.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-digester-1.8.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-io-2.4.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/avro-1.7.4.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/log4j-1.2.17.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hadoop-annotations-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/xmlenc-0.52.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-configuration-1.6.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/servlet-api-2.5.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jets3t-0.9.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jetty-6.1.26.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-codec-1.4.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jsch-0.1.42.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jsp-api-2.1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/zookeeper-3.4.6.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/httpclient-4.2.5.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/netty-3.6.2.Final.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jsr305-1.3.9.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jersey-core-1.9.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/junit-4.11.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-math3-3.1.1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-collections-3.2.1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jettison-1.1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/paranamer-2.3.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/httpcore-4.2.5.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/xz-1.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-httpclient-3.1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-el-1.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/htrace-core-3.0.4.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-logging-1.1.3.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/activation-1.1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/curator-framework-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/curator-recipes-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jersey-server-1.9.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-net-3.1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/guava-11.0.2.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-lang-2.6.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/asm-3.2.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/stax-api-1.0-2.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hadoop-auth-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jersey-json-1.9.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/gson-2.2.4.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/hadoop-nfs-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0-tests.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-io-2.4.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-el-1.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/htrace-core-3.0.4.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/guava-11.0.2.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/asm-3.2.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0-tests.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/commons-cli-1.2.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/commons-io-2.4.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jline-0.9.94.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/servlet-api-2.5.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-6.1.26.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/commons-codec-1.4.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-client-1.9.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jsr305-1.3.9.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-core-1.9.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/commons-collections-3.2.1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jettison-1.1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/xz-1.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/guice-3.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/activation-1.1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/javax.inject-1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-server-1.9.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/guava-11.0.2.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/asm-3.2.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-json-1.9.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-client-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-common-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-registry-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-api-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-common-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/junit-4.11.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/xz-1.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/guice-3.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/javax.inject-1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/asm-3.2.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-tests.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.6.0.jar:/usr/local/hadoop/hadoop-2.6.0/contrib/capacity-scheduler/*.jar
STARTUP_MSG:   build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r e3496499ecb8d220fba99dc5ed4c99c8f9e33bb1; compiled by 'jenkins' on 2014-11-13T21:10Z
STARTUP_MSG:   java = 1.8.0_60
************************************************************/
16/01/23 21:57:57 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
16/01/23 21:57:57 INFO namenode.NameNode: createNameNode [-format]
16/01/23 21:57:58 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
16/01/23 21:57:58 WARN common.Util: Path /usr/local/hadoop/hadoop-2.6.0/dfs/name should be specified as a URI in configuration files. Please update hdfs configuration.
16/01/23 21:57:58 WARN common.Util: Path /usr/local/hadoop/hadoop-2.6.0/dfs/name should be specified as a URI in configuration files. Please update hdfs configuration.
Formatting using clusterid: CID-9bd3d990-8e12-4581-b131-5021e35cfe78
16/01/23 21:57:58 INFO namenode.FSNamesystem: No KeyProvider found.
16/01/23 21:57:58 INFO namenode.FSNamesystem: fsLock is fair:true
16/01/23 21:57:58 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit=1000
16/01/23 21:57:58 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true
16/01/23 21:57:58 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
16/01/23 21:57:58 INFO blockmanagement.BlockManager: The block deletion will start around 2016 一月 23 21:57:58
16/01/23 21:57:58 INFO util.GSet: Computing capacity for map BlocksMap
16/01/23 21:57:58 INFO util.GSet: VM type       = 64-bit
16/01/23 21:57:58 INFO util.GSet: 2.0% max memory 966.7 MB = 19.3 MB
16/01/23 21:57:58 INFO util.GSet: capacity      = 2^21 = 2097152 entries
16/01/23 21:57:58 INFO blockmanagement.BlockManager: dfs.block.access.token.enable=false
16/01/23 21:57:58 INFO blockmanagement.BlockManager: defaultReplication         = 2
16/01/23 21:57:58 INFO blockmanagement.BlockManager: maxReplication             = 512
16/01/23 21:57:58 INFO blockmanagement.BlockManager: minReplication             = 1
16/01/23 21:57:58 INFO blockmanagement.BlockManager: maxReplicationStreams      = 2
16/01/23 21:57:58 INFO blockmanagement.BlockManager: shouldCheckForEnoughRacks  = false
16/01/23 21:57:58 INFO blockmanagement.BlockManager: replicationRecheckInterval = 3000
16/01/23 21:57:58 INFO blockmanagement.BlockManager: encryptDataTransfer        = false
16/01/23 21:57:58 INFO blockmanagement.BlockManager: maxNumBlocksToLog          = 1000
16/01/23 21:57:58 INFO namenode.FSNamesystem: fsOwner             = root (auth:SIMPLE)
16/01/23 21:57:58 INFO namenode.FSNamesystem: supergroup          = supergroup
16/01/23 21:57:58 INFO namenode.FSNamesystem: isPermissionEnabled = true
16/01/23 21:57:58 INFO namenode.FSNamesystem: HA Enabled: false
16/01/23 21:57:58 INFO namenode.FSNamesystem: Append Enabled: true
16/01/23 21:57:58 INFO util.GSet: Computing capacity for map INodeMap
16/01/23 21:57:58 INFO util.GSet: VM type       = 64-bit
16/01/23 21:57:58 INFO util.GSet: 1.0% max memory 966.7 MB = 9.7 MB
16/01/23 21:57:58 INFO util.GSet: capacity      = 2^20 = 1048576 entries
16/01/23 21:57:58 INFO namenode.NameNode: Caching file names occuring more than 10 times
16/01/23 21:57:59 INFO util.GSet: Computing capacity for map cachedBlocks
16/01/23 21:57:59 INFO util.GSet: VM type       = 64-bit
16/01/23 21:57:59 INFO util.GSet: 0.25% max memory 966.7 MB = 2.4 MB
16/01/23 21:57:59 INFO util.GSet: capacity      = 2^18 = 262144 entries
16/01/23 21:57:59 INFO namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.9990000128746033
16/01/23 21:57:59 INFO namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 0
16/01/23 21:57:59 INFO namenode.FSNamesystem: dfs.namenode.safemode.extension     = 30000
16/01/23 21:57:59 INFO namenode.FSNamesystem: Retry cache on namenode is enabled
16/01/23 21:57:59 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
16/01/23 21:57:59 INFO util.GSet: Computing capacity for map NameNodeRetryCache
16/01/23 21:57:59 INFO util.GSet: VM type       = 64-bit
16/01/23 21:57:59 INFO util.GSet: 0.029999999329447746% max memory 966.7 MB = 297.0 KB
16/01/23 21:57:59 INFO util.GSet: capacity      = 2^15 = 32768 entries
16/01/23 21:57:59 INFO namenode.NNConf: ACLs enabled? false
16/01/23 21:57:59 INFO namenode.NNConf: XAttrs enabled? true
16/01/23 21:57:59 INFO namenode.NNConf: Maximum size of an xattr: 16384
16/01/23 21:57:59 INFO namenode.FSImage: Allocated new BlockPoolId: BP-773392318-192.168.112.130-1453557479065
16/01/23 21:57:59 INFO common.Storage: Storage directory /usr/local/hadoop/hadoop-2.6.0/dfs/name has been successfully formatted.
16/01/23 21:57:59 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
16/01/23 21:57:59 INFO util.ExitUtil: Exiting with status 0
16/01/23 21:57:59 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at master1/192.168.112.130
************************************************************/
root@master1:/usr/local/hadoop/hadoop-2.6.0/bin#

启动hdfs: ./sbin/start-dfs.sh 打印出如下信息： 
root@master1:/usr/local/hadoop/hadoop-2.6.0/sbin# start-dfs.sh 
16/01/23 22:01:06 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [master1]
The authenticity of host 'master1 (192.168.112.130)' can't be established.
ECDSA key fingerprint is SHA256:ctbDQwKOmzr0Ixl43vm1akZAP360sbX4rHWk5FZ2gpo.
Are you sure you want to continue connecting (yes/no)? yes
master1: Warning: Permanently added 'master1,192.168.112.130' (ECDSA) to the list of known hosts.
master1: starting namenode, logging to /usr/local/hadoop/hadoop-2.6.0/logs/hadoop-root-namenode-master1.out
worker4: starting datanode, logging to /usr/local/hadoop/hadoop-2.6.0/logs/hadoop-root-datanode-worker4.out
worker1: starting datanode, logging to /usr/local/hadoop/hadoop-2.6.0/logs/hadoop-root-datanode-worker1.out
worker3: starting datanode, logging to /usr/local/hadoop/hadoop-2.6.0/logs/hadoop-root-datanode-worker3.out
worker2: starting datanode, logging to /usr/local/hadoop/hadoop-2.6.0/logs/hadoop-root-datanode-worker2.out
Starting secondary namenodes [master1]
master1: starting secondarynamenode, logging to /usr/local/hadoop/hadoop-2.6.0/logs/hadoop-root-secondarynamenode-master1.out
16/01/23 22:01:32 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

用命令jps验证master1上进程，启动了NameNode和SecondaryNameNode后台进程； 
root@master1:/usr/local/hadoop/hadoop-2.6.0/sbin# jps
4498 Jps
4379 SecondaryNameNode
4175 NameNode

在worker1上用jps验证，启动了DataNode后台进程； 
root@worker1:/usr/local/hadoop/hadoop-2.6.0/etc/hadoop# jps
2563 DataNode
2646 Jps

在master1上启动yarn:  ./sbin/start-yarn.sh 打印出如下信息： 
root@master1:/usr/local/hadoop/hadoop-2.6.0/sbin# start-yarn.sh
starting yarn daemons
starting resourcemanager, logging to /usr/local/hadoop/hadoop-2.6.0/logs/yarn-root-resourcemanager-master1.out
worker1: starting nodemanager, logging to /usr/local/hadoop/hadoop-2.6.0/logs/yarn-root-nodemanager-worker1.out
worker3: starting nodemanager, logging to /usr/local/hadoop/hadoop-2.6.0/logs/yarn-root-nodemanager-worker3.out
worker2: starting nodemanager, logging to /usr/local/hadoop/hadoop-2.6.0/logs/yarn-root-nodemanager-worker2.out
worker4: starting nodemanager, logging to /usr/local/hadoop/hadoop-2.6.0/logs/yarn-root-nodemanager-worker4.out

在master1上用jps验证，启动了ResourceManager后台进程； 
root@master1:/usr/local/hadoop/hadoop-2.6.0/sbin# jps
4821 Jps
4551 ResourceManager
4379 SecondaryNameNode
4175 NameNode

在worker1上用jps验证，启动了NodeManager后台进程； 
root@worker1:/usr/local/hadoop/hadoop-2.6.0/etc/hadoop# jps
2563 DataNode
2713 NodeManager
2829 Jps

关闭hdfs用命令./sbin/stop-dfs.sh
关闭yarn用命令./sbin/stop-yarn.sh
查看集群状态用命令./bin/hdfs dfsadmin -report
查看文件块组成： ./bin/hdf sfsck / -files -blocks
root@master1:/usr/local/hadoop/hadoop-2.6.0/bin# hdfs fsck / -files -blocks
16/01/23 22:13:46 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Connecting to namenode via http://master1:50070
FSCK started by root (auth:SIMPLE) from /192.168.112.130 for path / at Sat Jan 23 22:13:47 CST 2016
/ <dir>
Status: HEALTHY
Total size:    0 B
Total dirs:    1
Total files:    0
Total symlinks:        0
Total blocks (validated):    0
Minimally replicated blocks:    0
Over-replicated blocks:    0
Under-replicated blocks:    0
Mis-replicated blocks:        0
Default replication factor:    2
Average block replication:    0.0
Corrupt blocks:        0
Missing replicas:        0
Number of data-nodes:        4
Number of racks:        1
FSCK ended at Sat Jan 23 22:13:47 CST 2016 in 2 milliseconds

The filesystem under path '/' is HEALTHY
4.4 用浏览器访问集群
在master1上输入http://192.168.112.130:50070 或者master1:50070访问



用http://192.168.112.130:8088/查看RM的监控页面



到此为止，hadoop集群已经安装成功！！！

  • 大小: 47.6 KB
  • 大小: 134.3 KB
  • 大小: 85.3 KB
分享到：
| ubuntu 15.10虚拟机设置静态ip
评论
发表评论

您还没有登录,请您登录后再发表评论

相关推荐

    Hadoop2.6.0集群环境搭建

    1.linux系统:Ubuntu14.04 2.hadoop版本:hadoop-2.2.0 3.JDK版本:Jdk1.8.0_74

    Ubantu-hadoop2.6.0集群搭建

    ### Hadoop集群搭建知识点 #### 一、概述 Hadoop是一种能够处理大量数据的大规模分布式存储与计算平台，常用于构建大数据分析系统。Hadoop的核心组件包括HDFS（Hadoop Distributed File System）和MapReduce。在...

    CentOS 6.8 + Hadoop2.6.0集群环境搭建

    CentOS 6.8 + Hadoop2.6.0集群环境搭建指南。

    hadoop2.6.0集群搭建（个人笔记，亲测有效）

    hadoop2.6.0完全分布式搭建

    hadoop 2.6.0 安装包

    这个"hadop 2.6.0 安装包"是为了帮助用户搭建和配置Hadoop环境，特别适用于大数据处理和分析。 1. **Hadoop的核心组件** - **HDFS（Hadoop Distributed File System）**：分布式文件系统，负责数据的存储。HDFS...

    Centos7+hadoop2.6.0分布式集群搭建

    集群搭建的最后一步是启动Hadoop的各个服务，这包括格式化HDFS、启动HDFS的NameNode、DataNode、SecondaryNameNode以及YARN的ResourceManager和NodeManager。通过`jps`命令可以查看这些服务是否已经启动成功。 ### ...

    hadoop2.6.0插件+64位winutils+hadoop.dll

    Hadoop2.6.0是这个框架的一个重要版本，它包含了多项优化和改进，以提高系统的稳定性和性能。在这个压缩包中，我们关注的是与Windows环境相关的两个关键组件：Winutils和hadoop.dll。 首先，让我们详细了解一下...

    hadoop2.6.0.rar

    3. 解压Hadoop压缩包：使用`tar -zxvf hadoop2.6.0.tgz`命令解压。 4. 配置环境变量：在`~/.bashrc`或`~/.bash_profile`文件中设置HADOOP_HOME，并添加到PATH。 5. 配置Hadoop配置文件：修改`etc/hadoop/core-site....

    hadoop2.6.0版本hadoop.dll和winutils.exe

    在标题中提到的“hadoop2.6.0版本hadoop.dll和winutils.exe”是针对Windows环境下运行Hadoop的一些关键组件。 1. **Hadoop 2.6.0**: 这是Hadoop的一个主要版本，发布于2014年，带来了许多增强和改进。在Hadoop 2.x...

    Hadoop2.6.0版本windows环境搭建

    本文将详细介绍如何在Windows环境下搭建Hadoop2.6.0版本。首先，我们需要从指定的下载地址获取CDH（Cloudera Distribution Including Apache Hadoop）提供的Hadoop2.6.0-cdh5.13.0的压缩包，地址为：...

    hadoop-2.6.0.tar.gz&hadoop-2.6.0-cdh5.16.2.tar.gz

    用户可以通过解压此文件，编译安装来搭建自己的Hadoop环境，进行分布式计算和数据存储。这个版本还包含了其他相关工具，如Hadoop命令行工具、Hadoop守护进程等，用于管理和操作Hadoop集群。 而hadoop-2.6.0-cdh...

    hadoop.dll & winutils.exe For hadoop-2.6.0

    在Hadoop生态系统中，`hadoop.dll`和`winutils.exe`是两个关键组件，尤其对于Windows用户来说。本文将详细介绍这两个文件以及它们在Hadoop ...正确配置和使用这些文件，对于在Windows上搭建和管理Hadoop集群至关重要。

    hadoop2.6.0（winutils、hadoop.dll）

    4. **HDFS模拟器**: Hadoop2.6.0版本包含了HDFS的本地模拟器，使得开发者可以在单机的Windows环境下测试HDFS操作，而无需完整的分布式集群。这对于开发和调试Hadoop应用程序非常有用。 5. **安全认证**: winutils....

    eclipse-hadoop2.6.0插件.rar

    Eclipse Hadoop 2.6.0 插件是针对Hadoop开发的一款工具，它使得在Eclipse集成开发环境中管理、配置和调试Hadoop项目变得简单高效。这个插件不仅提高了开发人员的工作效率，还降低了操作Hadoop集群的复杂性。 Hadoop...

    hadoop2.6.0版本-hadoop-2.6.0.tar.gz

    这个压缩包“hadoop-2.6.0.tar.gz”包含了Hadoop 2.6.0版本的所有组件，是搭建Hadoop集群的关键组成部分。在这个版本中，Hadoop已经相当成熟，提供了许多改进和新特性，使得它在大数据处理领域更加高效和稳定。 在...

    hadoop2.6.0核心零件winutils.zip

    这个工具通常包含在`hadoop2.6.0`版本的核心组件中，并被放置在`HADOOP_HOME\bin`目录下。本文将深入探讨`winutils.exe`的作用、功能以及如何解决在Windows上运行Hadoop时遇到的相关问题。 一、`winutils.exe`简介 ...

    hadoop2.6.0伪分布安装

    - 创建 `/opt/yarn` 目录，并进入该目录使用 `tar xvf hadoop2.6.0.tar.gz` 解压文件。 3. **用户和组创建**： - 根据实际需要创建用户组（例如 `hadoop`），以及用户（如 `yarn`, `hdfs`, `mapred`）。 4. **...

    Hadoop 2.6.0+Hbase1.12+mahout0.9 集群搭建

    Hadoop 2.6.0+Hbase1.12+mahout0.9 集群搭建,自己写的，可以根据实际情况搭建伪分布式或者完全分布式。

    编译过的Hadoop-2.6.0.tar.gz

    《Hadoop-2.6.0在CentOS6.5上的编译与集群搭建详解》 Hadoop，作为开源的大数据处理框架，是大数据领域的核心工具之一。版本2.6.0是其发展历程中的一个重要里程碑，它带来了诸多性能优化和功能增强。在本文中，我们...

Magicbox
Global site tag (gtag.js) - Google Analytics