hadoop配置

所有的基本的配置，在下载下来的hadoop包中都有。hadoop-0.20.203.0\docs

 

1、下载hadoop，放到linux中

 

 

2、

 

修改以下配置文件。

 

conf/core-site.xml:

 

<configuration>

<property>

<name>fs.default.name</name>

<value>hdfs://localhost:9000</value>

</property>

</configuration>

 

--------------------------------------------------------------------------

conf/hdfs-site.xml:

 

<configuration>

<property>

<name>dfs.replication</name>

<value>1</value>

</property>

</configuration>

 

--------------------------------------------------------------------------

conf/mapred-site.xml:

 

<configuration>

<property>

<name>mapred.job.tracker</name>

<value>localhost:9001</value>

</property>

</configuration>

 

--------------------------------------------------------------------------

conf/hadoop-env.sh

 

# Set Hadoop-specific environment variables here.

 

# The only required environment variable is JAVA_HOME.  All others are

# optional.  When running a distributed configuration it is best to

# set JAVA_HOME in this file, so that it is correctly defined on

# remote nodes.

 

# The java implementation to use.  Required.

# export JAVA_HOME=/usr/lib/j2sdk1.5-sun

 

取消红色部分注释，然后修改我机器上的jdk目录

 

--------------------------------------------------------------------------

3、

配置 ssh

Now check that you can ssh to the localhost without a passphrase:

$ ssh localhost

If you cannot ssh to localhost without a passphrase, execute the following commands:

$ ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa

$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

6.3. Execution

Format a new distributed-filesystem:

$ bin/hadoop namenode -format

Start the hadoop daemons:

$ bin/start-all.sh

 

 

4、如果启动后，发现日志中有异常。那么重新格式化namenode，然后启动。

 

 

5、在linux中安装eclipse 。版本最好是3.2的 我用的是

eclipse-java-europa-winter-linux-gtk-x86_64.tar.gz

 

6、解压后，复制hadoop-0.20.203.0\contrib\eclipse-plugin\hadoop-eclipse-plugin-0.20.203.0.jar。到eclipse的plugins目录下。启动eclipse

 

7、切换到 map/reduce编辑模式

 

明天抓图再写