win xp搭建hadoop

win xp搭建hadoop

一 环境准备

Win xp

Cygwin1.7.9

Hadoop-0.20.2

Hbase-0.90.3

Zookeeper-3.3.2

jdk-6u26-windows-i586.exe

 

二 安装cygwin

双击setup.exe

因为我已经在别的机器安装过，所以有安装拷贝，选择第三个；如果不是，选择第一个从internet安装

推荐直接安装在根目录

选择安装文件，如果从internet下载，需要选择存放的位置：

弹出这个不管，确定即可

这里是开始安装，注意，默认是不安装ssh的，而我们需要安装

输入ssh，自动匹配

用鼠标点击一下net defult下面的，由skip改为安装的版本

 

三 配置ssh

 

双击桌面的Cygwin，打开终端

没配置ssh服务前，是连接不成功

Administrator@pc ~

$ ssh localhost

ssh: connect to host localhost port 22: Connection refused

 

输入ssh-host-config配置，一直输入yes，value输入netsec tty

Administrator@pc ~

$ ssh-host-config

*** Info: Generating /etc/ssh_host_key

*** Info: Generating /etc/ssh_host_rsa_key

*** Info: Generating /etc/ssh_host_dsa_key

*** Info: Generating /etc/ssh_host_ecdsa_key

*** Info: Creating default /etc/ssh_config file

*** Info: Creating default /etc/sshd_config file

*** Info: Privilege separation is set to yes by default since OpenSSH 3.3.

*** Info: However, this requires a non-privileged account called 'sshd'.

*** Info: For more info on privilege separation read /usr/share/doc/openssh/READ

ME.privsep.

*** Query: Should privilege separation be used? (yes/no) yes

*** Info: Note that creating a new user requires that the current account have

*** Info: Administrator privileges.  Should this script attempt to create a

*** Query: new local account 'sshd'? (yes/no) yes

*** Info: Updating /etc/sshd_config file

*** Info: Added ssh to C:\WINDOWS\system32\driversc\services

 

 

*** Warning: The following functions require administrator privileges!

 

*** Query: Do you want to install sshd as a service?

*** Query: (Say "no" if it is already installed as a service) (yes/no) yes

*** Query: Enter the value of CYGWIN for the daemon: [] netsec tty

 

*** Info: The sshd service has been installed under the LocalSystem

*** Info: account (also known as SYSTEM). To start the service now, call

*** Info: `net start sshd' or `cygrunsrv -S sshd'.  Otherwise, it

*** Info: will start automatically after the next reboot.

 

*** Info: Host configuration finished. Have fun!

 

 

启动服务

Administrator@pc ~

$ net start sshd

CYGWIN sshd 服务正在启动 .

CYGWIN sshd 服务已经启动成功。

 

连接自己和别人都成功

Administrator@pc ~

$ ssh localhost

Administrator@localhost's password:

 

Administrator@pc ~

$ ssh 130.51.38.219

The authenticity of host '130.51.38.219 (130.51.38.219)' can't be established.

ECDSA key fingerprint is c4:d0:48:2a:56:c5:22:26:7e:28:61:2b:6c:e3:c1:7a.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added '130.51.38.219' (ECDSA) to the list of known hosts.

Administrator@130.51.38.219's password:

Last login: Mon Aug  8 14:58:31 2011 from localhost

 

三 免密码ssh配置

比如，需要master免密码ssh到slave，则需要在master生成公钥放到slave上

Administrator@pc ~

$ ssh-keygen

Generating public/private rsa key pair.

Enter file in which to save the key (/home/Administrator/.ssh/id_rsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /home/Administrator/.ssh/id_rsa.

Your public key has been saved in /home/Administrator/.ssh/id_rsa.pub.

The key fingerprint is:

e6:15:b6:91:05:ce:6a:31:41:ad:00:14:cd:86:1b:b6 Administrator@pc

The key's randomart image is:

+--[ RSA 2048]----+

|   .+* .o....    |

|    + =  +.o     |

|   . = .o.B      |

|    E   .= +     |

|        S o      |

|       + .       |

|        .        |

|                 |

|                 |

+-----------------+

 

公钥生成在D:\cygwin\home\Administrator\.ssh，可以生成keys文件给s，也可以把里面的串给s，让s自己粘贴到自己的keys文件里。

Administrator@pc ~

$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

 

在master里修改C:\WINDOWS\system32\drivers\etc\hosts，增加slave的机器名和ip映射

改完ssh的配置记得重启！net stop sshd

 

四 安装java

建议更改为没有空格的路径：（jre也是）

 

五 安装hadoop

把下载的hadoop-0.20.2.tar.gz，hbase-0.90.3.tar.gz，zookeeper-3.3.2.tar.gz放到~目录（放到哪只是个人喜好），然后解压

 

六 配置hadoop

配置以下文件

hadoop-env.sh

core-site.xml

hdfs-site.xml

mapred-site.xml

hbase-env.sh

hbase-site.xml

zoo.cfg

 

把hbase的lib文件夹下的hadoop-core的jar替换为当前的版本。