【Jstorm-0.9.6.2基本安装】

 

一、下载

[root@hadoop0 bigdata]# ls

apache-flume-1.6.0-bin         apache-hive-2.0.1-bin.tar.gz    hbase-1.1.5-bin.tar.gz  jstorm-0.9.6.2.zip  sqoop-1.4.6                                 stomr096                  tomcat7

apache-flume-1.6.0-bin.tar.gz  apache-tomcat-7.0.69.zip      hbase-1.1.5  hive2.0                 kafka               sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz  taokeeper-monitor.tar.gz  zookeeper

 

 

 

二、解压缩

[root@hadoop0 bigdata]# unzip jstorm-0.9.6.2.zip 

Archive:  jstorm-0.9.6.2.zip

   creating: jstorm-0.9.6.2/

   creating: jstorm-0.9.6.2/data/

[root@hadoop0 jstorm-0.9.6.2]# ls

bin  conf  data  example  history.md  jstorm-client-0.9.6.2.jar  jstorm-client-extension-0.9.6.2.jar  jstorm-server-0.9.6.2.jar  jstorm-ui-0.9.6.2.war  lib  LICENSE  logs  README.md  RELEASE

 

 

三、安装

 

1）安装Server

配置$JSTORM_HOME/conf/storm.yaml

[root@hadoop0 conf]# cat storm.yaml

########### These MUST be filled in for a storm configuration

 storm.zookeeper.servers:

     - "localhost"

 

 storm.zookeeper.root: "/jstorm"

 

# %JSTORM_HOME% is the jstorm home directory

 storm.local.dir: "%JSTORM_HOME%/data"

 

 java.library.path: "/usr/local/lib:/opt/local/lib:/usr/lib"

 

 

 supervisor.slots.ports:

    - 6800

    - 6801

    - 6802

    - 6803

 

# Default disable user-define classloader

# If there are jar conflict between jstorm and application, 

# please enable it 

 topology.enable.classloader: false

 

# enable supervisor use cgroup to make resource isolation

# Before enable it, you should make sure:

#       1. Linux version (>= 2.6.18)

#       2. Have installed cgroup (check the file's existence:/proc/cgroups)

#       3. You should start your supervisor on root

# You can get more about cgroup:

#   http://t.cn/8s7nexU

 supervisor.enable.cgroup: false

 

 

### Netty will send multiple messages in one batch  

### Setting true will improve throughput, but more latency

 storm.messaging.netty.transfer.async.batch: true

 

### if this setting  is true, it will use disruptor as internal queue, which size is limited

### otherwise, it will use LinkedBlockingDeque as internal queue , which size is unlimited

### generally when this setting is true, the topology will be more stable,

### but when there is a data loop flow, for example A -> B -> C -> A

### and the data flow occur blocking, please set this as false

 topology.buffer.size.limited: true

 

### default worker memory size, unit is byte

 worker.memory.size: 2147483648

 

# Metrics Monitor

# topology.performance.metrics: it is the switch flag for performance 

# purpose. When it is disabled, the data of timer and histogram metrics 

# will not be collected.

# topology.alimonitor.metrics.post: If it is disable, metrics data

# will only be printed to log. If it is enabled, the metrics data will be

# posted to alimonitor besides printing to log.

 topology.performance.metrics: true

 topology.alimonitor.metrics.post: false

 

# UI MultiCluster

# Following is an example of multicluster UI configuration

 ui.clusters:

     - {

         name: "jstorm",

         zkRoot: "/jstorm",

         zkServers:

             [ "localhost"],

         zkPort: 2181,

       }

 

 

配置项：

 

storm.zookeeper.servers: 表示zookeeper 的地址，

nimbus.host: 表示nimbus的地址

storm.zookeeper.root: 表示JStorm在zookeeper中的根目录，当多个JStorm共享一个zookeeper时，需要设置该选项，默认即为“/jstorm”

storm.local.dir: 表示JStorm临时数据存放目录，需要保证JStorm程序对该目录有写权限

java.library.path: Zeromq 和java zeromq library的安装目录，默认"/usr/local/lib:/opt/local/lib:/usr/lib"

supervisor.slots.ports: 表示Supervisor 提供的端口Slot列表，注意不要和其他端口发生冲突，默认是68xx，而Storm的是67xx

topology.enable.classloader: false, 默认关闭classloader，如果应用的jar与JStorm的依赖的jar发生冲突，比如应用使用thrift9，但jstorm使用thrift7时，就需要打开classloader。建议在集群级别上默认关闭，在具体需要隔离的topology上打开这个选项。

 

2）安装UI

mv jstorm-0.9.6.2/jstorm-ui-0.9.6.2.war  tomcat7/webapps/

 

四、启动

[root@hadoop0 bigdata]# cd zookeeper/

[root@hadoop0 zookeeper]# ls

bin          dist-maven       LICENSE.txt           src

build.xml    docs             NOTICE.txt            zookeeper-3.4.6.jar

CHANGES.txt  ivysettings.xml  README_packaging.txt  zookeeper-3.4.6.jar.asc

conf         ivy.xml          README.txt            zookeeper-3.4.6.jar.md5

contrib      lib              recipes               zookeeper-3.4.6.jar.sha1

[root@hadoop0 zookeeper]# cd bin/

[root@hadoop0 bin]# ./zkServer.sh  start

JMX enabled by default

Using config: /opt/bigdata/zookeeper/bin/../conf/zoo.cfg

Starting zookeeper ... STARTED

[root@hadoop0 bin]# cd ../../jstorm-0.9.6.2/bin/

[root@hadoop0 bin]# ./jstorm nimbus &

[1] 22494

[root@hadoop0 bin]# ./jstorm supervisor  &

[2] 32600

[root@hadoop0 bin]# ./startup.sh 

Using CATALINA_BASE:   /opt/bigdata/tomcat7

Using CATALINA_HOME:   /opt/bigdata/tomcat7

Using CATALINA_TMPDIR: /opt/bigdata/tomcat7/temp

Using JRE_HOME:        /opt/jdk1.7.0_79

Using CLASSPATH:       /opt/bigdata/tomcat7/bin/bootstrap.jar:/opt/bigdata/tomcat7/bin/tomcat-juli.jar

Tomcat started.

[root@hadoop0 bin]# 

 

 

 

五、验证

进程

 

WEBUI刚启动

 

 

服务启动