Hadoop环境搭建

1.给hadoop用户赋予和root用户同样的权限

sudo vim /etc/sudoers

 

2.修改hostname

sudo vi /etc/hostname

 

3.修改hosts

sudo vi /etc/hosts

 

4.重启Ubuntu（重启之后，可以尝试ping hostname，比如ping hadoop-0000）

sudo reboot

 

5. Alt + P, put JDK & Hadoop

 

6. 解压JDK & Hadoop

sudo tar zxvf jdk-8u77-linux-x64.tar.gz -C app/
sudo tar zxvf hadoop-2.6.4.tar.gz -C app/

 

7.配置JAVA_HOME & HADOOP_HOME

sudo vi /etc/profile

#setup Java & Hadoop environment
export JAVA_HOME=/home/hadoop/app/jdk1.8.0_77
export HADOOP_HOME=/home/hadoop/app/hadoop-2.6.4
export PATH=$PATH:${JAVA_HOME}/bin:${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin

 

8.使环境变量生效

source /etc/profile

 

 9.查看Hadoop文件夹

 

10.配置Hadoop环境

 

修改Hadoop环境的用户和用户组

sudo chown -R hadoop:hadoop hadoop-2.6.4/

 

配置hadoop-env.sh

sudo vi hadoop-env.sh

export JAVA_HOME=/home/hadoop/app/jdk1.8.0_77

 

配置core-site.xml

sudo vi core-site.xml

<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://hadoop-0000:9000</value>
    </property>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>/home/hadoop/app/hadoop-2.6.4/data</value>
    </property>
</configuration>

 

配置hdfs-site.xml

sudo vi hdfs-site.xml

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

 

配置mapred-site.xml

cp mapred-site.xml.template mapred-site.xml
sudo vi mapred-site.xml

<configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
</configuration>

 

配置yarn-site.xml

sudo vi yarn-site.xml

<configuration>

<!-- Site specific YARN configuration properties -->
    <property>
        <name>yarn.resourcemanager.hostname</name>
        <value>hadoop-0000</value>
    </property>

    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
</configuration>

 

配置slaves

sudo vi slaves

 

格式化（格式化一次就行了，否则将导致DataNode无法启动）

hadoop namenode -format

 

启动start-dfs.sh

 

 

启动start-yarn.sh

 

配置Windows Hosts

 

输入http://hadoop-0000:50070/