hadoop集群简单搭建

准备三台机器

vm101(namenode)
vm102(datanode)
vm103(datanode)

都下载好hadoop（1.1.x版本）以及jdk目录如下

/root/hadoop
/root/jdk1.6.0_38

编辑三台机器的/etc/HOSTNAME分别为vm101,vm102,vm103，三台机器均配置/ets/hosts文件添加如下映射

192.168.1.101       vm101  vm101.site
192.168.1.102       vm102  vm102.site
192.168.1.103       vm103  vm103.site

在vm101上创建ssh-keygen -t rsa，将.ssh/id_rsa.pub追加到vm101,vm102,vm103的~/.ssh/authorized_keys里面，实现从vm101到vm101,vm102,vm103的自动登录（如果启动集群失败可能需要事先手动登录输入密码一次）

在vm101上编辑配置文件如下

#编辑conf/core-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
    <name>fs.default.name</name>
    <value>hdfs://192.168.1.101:9000</value>
 </property>
</configuration>


#编辑conf/mapred-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
  <property>
    <name>mapred.job.tracker</name>
    <value>hdfs://192.168.1.101:9001</value>
  </property>
</configuration>

#编辑conf/hdfs-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->
<configuration>
  <property>
    <name>dfs.replication</name>
    <value>3</value>
  </property>
  <property>
    <name>dfs.name.dir</name>
    <value>/root/opt/hadoop/var/namedir</value>
  </property>
  <property>
    <name>dfs.data.dir</name>
    <value>/root/opt/hadoop/var/datadir</value>
  </property>
</configuration>
#编辑conf/hadoop-env.sh
export JAVA_HOME=/root/jdk1.6.0_38
#编辑conf/masters
vm101
#编辑conf/slaves
vm102
vm103

将conf/*拷贝到vm102，vm103两台机器。

在vm101上执行

bin/hadoop namenode -format

bin/start-all.sh

查看运行结果（namenode正常运行，datanode正常运行）

vm101:~/hadoop # jps
25091 NameNode
15110 DataNode
15514 TaskTracker
25284 SecondaryNameNode
25821 Jps
25374 JobTracker

vm103:~ # jps
11150 TaskTracker
11043 DataNode
11468 Jps