Hadoop,Hbase,Zookeeper在虚拟机单节点中的整合

    近日整合Hadoop,Hbase,Zookeeper时遇到不少问题，经过一番苦战后终于整合成功，现将整合中遇到的一些问题整理一下。

前置说明：
版本：
JDK: 1.6.*
Hadoop: 0.20.2
Hbase: 0.90.4
Zookkeeper: 3.4.3
centOS: CentOS release 5.3

节点数： 1 （虚拟机）

安装配置：
一、安装JDK

二、配置host
     修改配置 /etc/hosts 如下：
         192.168.0.23   hadoopName
     此hostname为hadoop、Hbase、Zookeeper配置文件中共同使用。这一步非常重要。也会涉及到windows中eclipse runonhadoop插件的运行。

二、安装Hadoop
      Hadoop安装比较容易，请参考《hadoop权威指南》、陆嘉恒版《hadoop实战》。

三、安装Zookeeper
      运行模式为：集群伪分布模式
      1、解压Zookeeper，将$ZK_INSTALL/bin加入环境变量。
      2、拷贝$ZK_INSTALL/conf下的zoo.example.cfg，分别复制为zoo1.cfg、zoo2.cfg、zoo3.cfg
      3、配置zoo*.cfg：
      zoo1.cfg

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
dataDir=/usr/local/zookeeper-3.4.3/snapshot/d_1
# the port at which the clients will connect
clientPort=2181
#the location of the log file
dataLogDir=/usr/local/zookeeper-3.4.3/logs
server.[color=red]1[/color]=hadoopName:2887:3887
server.[color=red]2[/color]=hadoopName:2888:3888
server.[color=red]3[/color]=hadoopName:2889:3889

      zoo2.cfg

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
dataDir=/usr/local/zookeeper-3.4.3/snapshot/d_2
# the port at which the clients will connect
clientPort=2182
#the location of the log file
dataLogDir=/usr/local/zookeeper-3.4.3/logs
server.[color=red]1[/color]=hadoopName:2887:3887
server.[color=red]2[/color]=hadoopName:2888:3888
server.[color=red]3[/color]=hadoopName:2889:3889

      zoo3.cfg

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
dataDir=/usr/local/zookeeper-3.4.3/snapshot/d_3
# the port at which the clients will connect
clientPort=2183
#the location of the log file
dataLogDir=/usr/local/zookeeper-3.4.3/logs
server.[color=red]1[/color]=hadoopName:2887:3887
server.[color=red]2[/color]=hadoopName:2888:3888
server.[color=red]3[/color]=hadoopName:2889:3889

      4、在zoo*.cfg中的dataDir指定的目录下，新建myid文件。
     例如：$ZK_INSTALL/snapshot/d_1下，新建myid。在myid文件中输入1。表示为server.1。
           如果为snapshot/d_2，则myid文件中的内容为 2，依此类推。

      5、如果在zoo*.cfg中配置了
     

 
       #the location of the log file
       dataLogDir=/usr/local/zookeeper-3.4.3/[color=red]logs[/color]
       

       如果$ZK_INSTALL下不存在logs目录，也会报错。所以需要手动创建logs文件夹。
       报错异常信息如下：
      

2012-02-26 00:06:47,364 [myid:] - INFO  [main:QuorumPeerConfig@101] - Reading configuration from: ../conf/zoo3.cfg
2012-02-26 00:06:47,372 [myid:] - ERROR [main:QuorumPeerMain@85] - Invalid config, exiting abnormally
org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Error processing ../conf/zoo3.cfg
        at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:121)
        at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:101)
        at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
Caused by: [color=red]java.lang.IllegalArgumentException: dataLogDir /usr/local/zookeeper-3.4.3/logs is missing.[/color]
        at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:247)
        at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:117)
        ... 2 more
Invalid config, exiting abnormally
       

   
       6、现在就可以启动zookeeper了：
       zkServer.sh start zoo1.cfg
       zkServer.sh start zoo2.cfg
       zkServer.sh start zoo3.cfg
       然后输入jps，查看进程：
       [root@hadoopname snapshot]# jps
4208 SecondaryNameNode
4587 QuorumPeerMain
4498 QuorumPeerMain
4273 JobTracker
4014 NameNode
4106 DataNode
4617 Jps
4547 QuorumPeerMain
4424 TaskTracker
 
     有3个QuorumPeerMain进程启动了，zookeeper启动成功。还可以使用zookeeper自带的四字命令进行启动成功的测试：
     [root@hadoopname snapshot]# echo ruok | nc hadoopName 2181
      imok
     返回了imok，证明启动成功。如果什么都没有返回，证明启动失败。

四、安装Hbase
      1、解压Hbase后，将$HBASE_INSTALL/bin加入环境变量。
      2、配置$HBASE_INSTALL/conf/hbase-env.sh：
         export JAVA_HOME=$YOUR_JAVA_HOME
         # Tell HBase whether it should manage it's own instance of Zookeeper or not.
         export HBASE_MANAGES_ZK=false    <==默认为true，我们不使用hbase自带的zookeeper实例。

      3、配置hbase-site.xml

<configuration>
<property>
        <name>hbase.rootdir</name>
        <value>hdfs://[color=red]hadoopName[/color]:9000/hbase</value>
</property>
<property>
        <name>hbase.cluster.distributed</name>
        <value>true</value>
</property>
<property>
        <name>dfs.replication</name>
        <value>1</value>
</property>
<property>
        <name>hbase.zookeeper.quorum</name>
        <value>[color=red]hadoopName[/color]</value>
</property>
<property>
        <name>zookeeper.session.timeout</name>
        <value>60000</value>
</property>

      5、将$HBASE_INSTALL/lib下的hadoop-core-0.20-append-r1056497.jar删除，替换为$HADOOP_INSTALL/hadoop-0.20.2-core.jar
      如果不替换，则在调用start-hbase.sh时HMaster无法启动。
      报错异常如下：
     

      12/02/26 00:35:02 INFO zookeeper.ClientCnxn: Session establishment complete on server hadoopname/192.168.0.23:2181, sessionid = 0x135b544b002000f, negotiated timeout = 40000
12/02/26 00:35:02 INFO client.HConnectionManager$HConnectionImplementation: ZooKeeper available but no active master location found
12/02/26 00:35:02 INFO client.HConnectionManager$HConnectionImplementation: getMaster attempt 0 of 1 failed; no more retrying.
org.apache.hadoop.hbase.MasterNotRunningException
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:357)
        at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:94)
        at org.apache.hadoop.hbase.master.HMasterCommandLine.stopMaster(HMasterCommandLine.java:163)
        at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:104)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
        at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1078)
12/02/26 00:35:02 ERROR master.HMasterCommandLine: Master not running
      

      ps：在hbase官方说明中有另外一种方法，小生暂时未试验过。http://hbase.apache.org/book.html#hadoop

      6、这时我们再启动start-hbase.sh，就会成功啦！
[root@hadoopname conf]# start-hbase.sh
starting master, logging to /usr/local/hbase-0.90.4/logs/hbase-root-master-hadoopname.out
hadoopName: starting regionserver, logging to /usr/local/hbase-0.90.4/logs/hbase-root-regionserver-hadoopname.out
[root@hadoopname conf]# jps
5004 HMaster
5101 HRegionServer
4208 SecondaryNameNode
5181 Jps
4587 QuorumPeerMain
4498 QuorumPeerMain
4273 JobTracker
4014 NameNode
4106 DataNode
4547 QuorumPeerMain
4424 TaskTracker

     至此，hadoop、hbase、zookeeper在虚拟机下的整合就完毕了。

总结：
     从整合的过程中可以感受到，hadoop的问题较少，而hbase和zk问题较多。比如zk的zoo.cfg配置参数中，dataDir指定的目录如果未创建，则zk会自动创建；但是dataLogDir参数指定的路径却不会自动创建。
     最后，如果本文中有什么遗漏或不正确的地方，还请各位朋友斧正。在下感激不尽！

评论

2 楼 小才纸 2016-03-08  

mark

1 楼 di1984HIT 2013-02-18  

嗯，记录一下。