storm集群安装

 

环境要求

centos 6.8/JDK 1.6+/java -version/Python 2.6.6+/python -V/ZooKeeper3.4.5+/apache-storm-0.9.4

环境变量

 

export STORM_HOME=/opt/sgb/apache-storm-0.9.4
export ZOOKEEPER_HOME=/opt/sgb/zookeeper-3.4.6
export JAVA_HOME=/usr/java/jdk1.7.0_67
export PATH=$PATH:$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$STORM_HOME/bin

 1、单机版

 

 

$ ./bin/storm --help
下面分别启动ZooKeeper、Nimbus、UI、supervisor、logviewer
$ ./bin/storm dev-zookeeper >> ./logs/zk.out 2>&1 &
$ ./bin/storm nimbus >> ./logs/nimbus.out 2>&1 &
$ ./bin/storm ui >> ./logs/ui.out 2>&1 &
$ ./bin/storm supervisor >> ./logs/supervisor.out 2>&1 &
$ ./bin/storm logviewer >> ./logs/logviewer.out 2>&1 &

 2、分布式版本

 

 

各节点分配：
         Nimbus    Supervisor   Zookeeper
node1      1                       1
node2                 1            1
node3                 1            1

 1、/opt/sgb/apache-storm-0.9.4/conf/storm.yaml

 

storm.zookeeper.servers:
     - "sto1"
     - "sto2"
     - "sto3"
storm.local.dir: "/opt/sgb/storm"
ui.port: 18080
nimbus.host: "sto1"
supervisor.slots.ports:
     - 6700
     - 6701
     - 6702
     - 6703

2、（分发代码）集群其他服务器

3、启动ZooKeeper集群

 

node1上启动Nimbus
$ ./bin/storm nimbus >> ./logs/nimbus.out 2>&1 &
$ tail -f logs/nimbus.log
$ ./bin/storm ui >> ./logs/ui.out 2>&1 &
$ tail -f logs/ui.log

节点node2和node3启动supervisor，按照配置，每启动一个supervisor就有了4个slots
$ ./bin/storm supervisor >> ./logs/supervisor.out 2>&1 &
$ tail -f logs/supervisor.log
（当然node1也可以启动supervisor）

 4、集群drpc

修改
$ vi conf/storm.yaml
drpc.servers:
    - "sto1"

(根据版本选择)storm.thrift.transport: "org.apache.storm.security.auth.plain.PlainSaslTransportPlugin"

 分发配置storm.yaml文件给其他节点

 

主节点启动  drpc

 storm drpc >> ./logs/drpc.out 2>&1 &

 storm.yaml配置汇总

storm.zookeeper.servers:
     - "sto1"
     - "sto2"
     - "sto3"
storm.local.dir: "/opt/sgb/storm"
ui.port: 18080
nimbus.host: "sto1"
supervisor.slots.ports:
     - 6700
     - 6701
     - 6702
     - 6703
drpc.servers:
     - "sto1"