hadoop 分布式系统安装与部署

最近公司要用分布式系统，所以现在hadoop部署，因为只有自己部署才能更好的理解系统的架构。

linux服务器：

cd /opt

wget http://labs.renren.com/apache-mirror/hadoop/core/hadoop-0.18.3/hadoop-0.18.3.tar.gz 获取最近版的hadoop文件

[root@localhost opt]# wget http://labs.renren.com/apache-mirror/hadoop/core/hadoop-0.18.3/hadoop-0.18.3.tar.gz
--16:35:52--  http://labs.renren.com/apache-mirror/hadoop/core/hadoop-0.18.3/hadoop-0.18.3.tar.gz
Resolving labs.renren.com... 123.129.232.121
Connecting to labs.renren.com|123.129.232.121|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 30705253 (29M) [application/x-gzip]
Saving to: `hadoop-0.18.3.tar.gz'

100%[========================================================================>] 30,705,253  2.06M/s   in 14s    

16:36:06 (2.06 MB/s) - `hadoop-0.18.3.tar.gz' saved [30705253/30705253]

[root@localhost opt]# ls
hadoop-0.18.3.tar.gz

 下载速度还是蛮快的

  解压下载后的文件

 tar zxvf hadoop-0.18.3.tar.gz

 将解压后的文件 放到hadoop文件夹里

mv hadoop-0.18.3 hadoop

 

chown -R hadooper:hadooper hadoop
对目前目录下的所有档案与子目录进行相同的拥有者变更(即以递回的方式逐个变更)

 设置环境变量：

[root@localhost conf]# cat >>hadoop-env.sh <<EOF
> export JAVA_HOME=/user/lib/jvm/java-6-sun
> export HADOOP_HOME=/opt/hadoop
> expott HADOOP_CONF_DIR=/opt/hadoop/conf
> EOF
或者：
export JAVA_HOME=/user/lib/jvm/java-6-sun
     60 export HADOOP_HOME=/opt/hadoop
     61 export HADOOP_CONF_DIR=/opt/hadoop/conf 

 写配置文件：

[root@localhost hadoop]# vi conf/hadoop-site.xml 
<configuration>
      2 <property>
      3 <name>com.mixian.hadoop</name>
      4 <value>hdfs://xx:9000</value>
      5 <description>default file system for NDFS</description>
      6 </property>
      7 <property>
      8  <name>mapred.job.tracker</name>
      9  <value>hdfs://xx:9001</value>
     10  <description>The host:port that job tracker runsat.</description>
     11 </property>
     12 </configuration>

 然后再格式化HDFS：

[root@localhost hadoop]# bin/hadoop namenode -format
bin/hadoop: line 234: /user/lib/jvm/java-6-sun/bin/java: No such file or directory
bin/hadoop: line 264: /user/lib/jvm/java-6-sun/bin/java: No such file or directory
bin/hadoop: line 264: exec: /user/lib/jvm/java-6-sun/bin/java: cannot execute: No such file or directory
[root@localhost hadoop]# 

 error:尴尬 发现测试服务器没有装过jdk

 

先去吃饭 回来写