大数据系列7：Storm – 流计算

wget http://download.zeromq.org/zeromq-2.1.7.tar.gz

tar -xzvf zeromq-2.1.7.tar.gz

cd zeromq-2.1.7

sudo apt-get install gcc

sudo apt-get install g++

sudo apt-get install libuuid-dev

./configure

sudo apt-get install make

make

sudo make install

 

sudo apt-get install git

git clone https://github.com/nathanmarz/jzmq.git

cd jzmq

sudo apt-get install pkg-config

./autogen.sh

./configure

touch src/classdist_noinst.stamp

cd src
javac -d . org/zeromq/*.java
cd ..

make

sudo make install

 

wget https://dl.dropboxusercontent.com/s/fl4kr7w0oc8ihdw/storm-0.8.2.zip

sudo apt-get install unzip

unzip storm-0.8.2.zip

cd storm-0.8.2

sudo vi /etc/profile

增加：

export STORM_HOME=/home/ysc/storm-0.8.2

export PATH=$PATH:$STORM_HOME/bin

       source /etc/profile

vi conf/storm.yaml

增加：

storm.zookeeper.servers:

     - "host001"

       nimbus.host: "host001"

       storm.local.dir: "/home/ysc/storm"

supervisor.slots.ports:

    - 6700

    - 6701

    - 6702
    -6703

       启动主节点

storm nimbus &

启动从节点

    storm supervisor &

启动WEB服务

       storm ui &

       访问：http://host001:8080

 

       git clone https://github.com/nathanmarz/storm-starter.git

       cd storm-starter

       sudo apt-get install maven2

       vi m2-pom.xml

       改变下面两个依赖的版本为3.0.3

                <dependency>

                       <groupId>org.twitter4j</groupId>

                       <artifactId>twitter4j-core</artifactId>

                       <version>3.0.3</version>

                </dependency>

                <dependency>

                       <groupId>org.twitter4j</groupId>

                       <artifactId>twitter4j-stream</artifactId>

                       <version>3.0.3</version>

                </dependency>

       mvn -f m2-pom.xml package

      

       cp m2-pom.xml pom.xml

       mvn eclipse:eclipse

       导入eclipse

       Java Build Path -> Add Variable… ->Configure Variables… -> New… -> Name: M2_REPO Path:maven存储库路径

      

       本地运行wordcount:

       storm jar target/storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jarstorm.starter.WordCountTopology

       集群运行wordcount:

       storm jartarget/storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jarstorm.starter.WordCountTopology  wordcount

 

 

Storm相关框架研究交流群，有兴趣的请加Q群：321914133

 

 

 

APDPlat旗下十大开源项目