S4-与ZooKeeper结合部署

AliKevin 写道

   本系列文章不涉及过多的S4的理论内容，因为s4论文中描述相当清楚（我认为我实在是说的不会比论文中更清楚了：）呵呵），论文信息请看论文http://dl.iteye.com/topics/download/704e5924-0dd8-34df-b44f-2efbc91de071

  
    S4的的运行模式有两种Static和Dynamic，期之前我们所做的学习完全基于s4的static模式。static模式不能对s4的集群进行智能的负载均衡，这对于分布式的大数量计算能力是一个不好的消息哦：），幸运的是S4可以借助现有的ZooKeeper智能的管理S4的集群：），接下来我们看看如何利用zookeeper和s4结合部署。

一、ZooKeeper的简单部署
    我们这里只是简单介绍s4与ZooKeeper的结合，不涉及到ZooKeeper的集群，这里试只是单节点启动ZooKeeper。
a.[下载ZooKeeper|http://labs.renren.com/apache-mirror//zookeeper/zookeeper-3.3.3/zookeeper-3.3.3.tar.gz]
b.安装ZooKeeper
将下载的ZooKeeper解压到你的安装目录，这里目录是/kevin/zookeeper-3.3.3,执行如下命令：

root@master:/# cd /kevin/zookeeper-3.3.3/conf
root@master:/kevin/zookeeper-3.3.3/conf# cp zoo.cfg.bak zoo.cfg
root@master:/kevin/zookeeper-3.3.3/conf# vi zoo.cfg

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
dataDir=/kevin/zookeeper-3.3.3/data ---> 修改到你的指定目录
# the port at which the clients will connect
clientPort=2181--->设置你要监听的端口（大于1024）

c.启动ZooKeeper实例

root@master:/kevin/zookeeper-3.3.3/conf# cd /kevin/zookeeper-3.3.3/bin/
root@master:/kevin/zookeeper-3.3.3/bin# ./zkServer.sh start
root@master:/kevin/zookeeper-3.3.3/bin# ./zkServer.sh start
JMX enabled by default
Using config: /kevin/zookeeper-3.3.3/bin/../conf/zoo.cfg
Starting zookeeper ...
STARTED
....
....


d.验证ZooKeeper的实例启动情况

root@master:/kevin/zookeeper-3.3.3/bin# cd /
root@master:/# cd /kevin/zookeeper-3.3.3/bin
root@master:/kevin/zookeeper-3.3.3/bin# ./zkCli.sh -server localhost:2181
Connecting to localhost:2181
...
...
WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0]
[zk: localhost:2181(CONNECTED) 29] ls /
[zookeeper]
[zk: localhost:2181(CONNECTED) 30]

如上显示证明我们已经启动的ZooKeeper实例，当前实例说管理的节点是跟目录下有个zookeeper节点。

二、将S4节点告知ZooKeeper

a.将S4集群信息告知ZooKeeper

root@master:~# $S4_IMAGE/scripts/task-setup.sh localhost:2181 clean setup $S4_IMAGE/s4-core/conf/dynamic/clusters.xml
RUNNING /usr/jdk1.6.0_26/bin/java -classpath /kevin/s4/build/s4-image/s4-core/lib/gson-1.6.jar:/kevin/s4/build/s4-image/s4-core/lib/junit-3.8.1.jar:/kevin/s4/build/s4-image/s4-core/lib/spring-2.5.6.jar:/kevin/s4/build/s4-image/s4-core/lib/S4WordCount.jar:/kevin/s4/build/s4-image/s4-core/lib/jakarta-regexp-1.4.jar:/kevin/s4/build/s4-image/s4-core/lib/bcel-5.2.jar:/kevin/s4/build/s4-image/s4-core/lib/log4j-1.2.15.jar:/kevin/s4/build/s4-image/s4-core/lib/kryo-1.01.jar:/kevin/s4/build/s4-image/s4-core/lib/json-20090211.jar:/kevin/s4/build/s4-image/s4-core/lib/s4-driver-0.3.0.jar:/kevin/s4/build/s4-image/s4-core/lib/mail-1.4.jar:/kevin/s4/build/s4-image/s4-core/lib/activation-1.1.jar:/kevin/s4/build/s4-image/s4-core/lib/flexjson-2.1.jar:/kevin/s4/build/s4-image/s4-core/lib/minlog-1.2.jar:/kevin/s4/build/s4-image/s4-core/lib/zookeeper-3.3.1.jar:/kevin/s4/build/s4-image/s4-core/lib/s4-example-driver-0.3.0.jar:/kevin/s4/build/s4-image/s4-core/lib/asm-3.2.jar:/kevin/s4/build/s4-image/s4-core/lib/commons-cli-1.2.jar:/kevin/s4/build/s4-image/s4-core/lib/commons-jexl-1.1.jar:/kevin/s4/build/s4-image/s4-core/lib/jline-0.9.94.jar:/kevin/s4/build/s4-image/s4-core/lib/s4-core-0.3.0.jar:/kevin/s4/build/s4-image/s4-core/lib/s4-comm-0.3.0.jar:/kevin/s4/build/s4-image/s4-core/lib/reflectasm-0.8.jar:/kevin/s4/build/s4-image/s4-core/lib/commons-logging-1.1.1.jar: io.s4.comm.tools.TaskSetupApp localhost:2181 clean setup /kevin/s4/build/s4-image/s4-core/conf/dynamic/clusters.xml
log4j:WARN Continuable parsing error 1 and column 70
log4j:WARN Document root element "log4j:configuration", must match DOCTYPE root "null".
log4j:WARN Continuable parsing error 1 and column 70
log4j:WARN Document is invalid: no grammar found.
log4j:WARN No appenders could be found for logger (io.s4.comm.core.DefaultWatcher).
log4j:WARN Please initialize the log4j system properly.
sTimeout=null
sTimeout=null
root@master:~#

b.检查ZooKeeper接管S4后的节点情况

root@master:/kevin/zookeeper-3.3.3/bin# ./zkCli.sh -server localhost:2181
Connecting to localhost:2181
...
...
WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0] ls /
[client-adapter, zookeeper, s4]
[zk: localhost:2181(CONNECTED) 1] ls /s4/s4
[process, task]
[zk: localhost:2181(CONNECTED) 5] ls /client-adapter/s4
[process, task]
[zk: localhost:2181(CONNECTED) 7]                 


如上信息证明我们已经将S4的集群信息告知给ZooKeeper了：）

三、S4实例验证

    与ZooKeeper结合要以Dynamic方式启动S4，故要修改S4的启动sh.当然修改方式有好多种（命令参数或硬性修改）这里只强调是要Dynamic模式：）
a.启动S4和Adapdter

root@slave:~# $S4_IMAGE/scripts/start-s4.sh dynamic -r client-adapter
...

appName=s4
dequeuer number: 6
sTimeout=5000
[]
[/kevin/s4/build/s4-image/s4-apps/s4-example-speech02/s4-example-speech02-conf.xml]
sTimeout=5000
sTimeout=5000
Adding processing element with bean name eventCatcher, id eventCatcher
adding pe: io.s4.example.speech01.SentenceReceiverPE@d58939
Using ConMapPersister ..
Adding processing element with bean name rerouteSentencePE, id rerouteSentencePE
adding pe: io.s4.processor.ReroutePE@93df2c
Using ConMapPersister ..
Adding processing element with bean name rerouteSpeechPE, id rerouteSpeechPE
adding pe: io.s4.processor.ReroutePE@3a1834
Using ConMapPersister ..
Adding processing element with bean name sentenceJoinPE, id sentenceJoinPE
adding pe: io.s4.processor.JoinPE@190a0d6
Using ConMapPersister ..


root@slave:~# $S4_IMAGE/scripts/run-client-adapter.sh -s client-adapter -g s4 -d $S4_IMAGE/s4-core/conf/dynamic/client-stub-conf.xml dynamic
...
appName=client-adapter
dequeuer number: 12
sTimeout=null
Adding InputStub genericStub
Adding OutputStub genericStub
sTimeout=null


b.执行测试和查看服务端输出

测试类：
root@slave:~# head -10 $S4_IMAGE/s4-example-testinput/speeches.txt | sh $S4_IMAGE/s4-tools-loadgenerator/scripts/generate-load.sh -r 2 -a localhost:2334 -d 1500 -
[: 27: ==: unexpected operator
[: 57: xlocalhost:2334: unexpected operator
[: 61: x2: unexpected operator
[: 65: x1500: unexpected operator
[: 69: x: unexpected operator
{0=io.s4.tools.loadgenerator.LoadGenerator$EventTypeInfo@60420f, 1=io.s4.tools.loadgenerator.LoadGenerator$EventTypeInfo@19106c7, 2=io.s4.tools.loadgenerator.LoadGenerator$EventTypeInfo@540408}
Emitted 9 events
服务端：
Using ConMapPersister ..
Sentence is 'Four score and seven years ago our fathers brought forth on this continent a new nation, conceived in liberty and dedicated to the proposition that all men are created equal.', location gettysburg, pa, us
Sentence is 'Four score and seven years ago our fathers brought forth on this continent a new nation, conceived in liberty and dedicated to the proposition that all men are created equal.', location gettysburg, pa, us
Sentence is 'Now we are engaged in a great civil war, testing whether that nation or any nation so conceived and so dedicated can long endure.', location gettysburg, pa, us
Sentence is 'Now we are engaged in a great civil war, testing whether that nation or any nation so conceived and so dedicated can long endure.', location gettysburg, pa, us
Sentence is 'We are met on a great battlefield of that war.', location gettysburg, pa, us
Sentence is 'We are met on a great battlefield of that war.', location gettysburg, pa, us
Sentence is 'We have come to dedicate a portion of that field as a final resting-place for those who here gave their lives that that nation might live.', location gettysburg, pa, us
Sentence is 'We have come to dedicate a portion of that field as a final resting-place for those who here gave their lives that that nation might live.', location gettysburg, pa, us


OK，如上信息证明我们联合ZooKeeper部署成功了，在使用的角度对客户是透明的，我们感性上感觉不到ZooKeeper的存在，而在S4内部则发生了根本的变化，当static模式时候S4的事件流处理由StaticTaskManager处理，而当我们以
dynamic 模式结合ZooKeeper运行S4时候，S4内部会应用ZkTaskManager管理事件流的处理。ZkTaskManager会利用ZooKeeper强大的集群管理能力，实现负载均衡等高效率的集群管理！：）

评论

3 楼 xyxiaoyou 2012-04-13  

请问LZ，S4的adapter集群、S4集群以及zookeeper集群之间是什么关系呢？ 如何配置，才能使得adapter集群和S4集群通信

2 楼 AliKevin2011 2011-11-02  

1

denghankun 写道

请问Lz，如果需要对zookeeper进行集群配置，应该如果把zookeeper集群配置到S4上

  1.zookeeper的集群配置是独立的，与S4没有必然联系。
  2.如果要s4与zookeeper集群进行集成只需要告知zookeeper集群要管理的s4集群信息（在s4的clusters.xml配置），只需要执行S4提供的ZkTaskSetup工具类即可，这个类会告知zookeeper关于S4信息。这样在S4运行期间就会被zookeeper进行管理了。
  

1 楼 denghankun 2011-10-27  

请问Lz，如果需要对zookeeper进行集群配置，应该如果把zookeeper集群配置到S4上