RocketMQ 3.2.6安装以及测试 -

gjp014

浏览: 272209 次
性别:
来自: 天津

最近访客更多访客>>

Cyen

maxer025

bing_it

dluccoo

博主相关

博客

微博

相册

留言

关于我

文章分类

社区版块

存档分类

RocketMQ 3.2.6安装以及测试

博客分类：

分布式技术

rocketmq

下载 alibaba-rocketmq-3.2.6.tar.gz
下载页面：https://github.com/alibaba/RocketMQ/releases
下载地址：
https://github.com/alibaba/RocketMQ/releases/download/v3.2.6/alibaba-rocketmq-3.2.6.tar.gz

解压
tar -xvf alibaba-rocketmq-3.2.6.tar.gz

需要jdk1.6以上
设置环境变量
vi .bash_profile

export JAVA_HOME=/opt/jdk6
export JRE_HOME=/opt/jdk6/jre
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib:CLASSPATH
export ROCKETMQ_HOME=/opt/mq/alibaba-rocketmq
export NAMESRV_ADDR=127.0.0.1:9876

启动
cd alibaba-rocketmq/bin
./play.sh

启动完成后
jps
会有NamesrvStartup和BrokerStartup两个java进程

如果没有问题，停止服务
cd alibaba-rocketmq/bin
sh mqshutdown broker
sh mqshutdown namesrv

二.设置防火墙：
默认启动用会使用3个端口 9876,10911,10912
分别代表名称服务端口，broker端口，broker ha端口。
ha端口haListenPort表示Master监听Slave请求的端口，默认为服务端口+1

添加到/etc/sysconfig/iptables的filter表INPUT链中

-A INPUT -m state --state NEW -m tcp -p tcp --dport 9876 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 10911 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 10912 -j ACCEPT

添加完成后重启防火墙
service iptables restart

四.获取broker默认配置
cd alibaba-rocketmq/bin
sh mqbroker -m

启动时如何加载配置文件呢
1.生成默认的配置模板
sh mqbroker -m > broker.p
2.修改配置
注意以下两个参数
storePathRootDir=/opt/mq/rocketmq/store
storePathCommitLog=/opt/mq/rocketmq/store/commitlog
默认commitlog是存放在用户主目录的下的store/commitlog目录下

namesrvAddr=127.0.0.1:9876
名称服务器地址，可以在命令行通过-n传入，多个名称服务器用;隔开

brokerIP1=192.168.6.57
根据实际情况填写，默认启动时自动识别,一般用于多网卡识别错误，手工配置。

brokerName=peteccBrkMaster
broker名称，默认主机名，我们可以改下peteccBrkMaster

listenPort=10911
默认监听端口

brokerId=0
0表示master,>0表示slave

autoCreateTopicEnable=true
是否自动创建topic,线上环境建议关闭

deleteWhen=04
删除文件的时间点，默认凌晨4点

fileReservdTime=48
文件保留时间，默认48小时

3.加载配置
nohup sh mqbroker -c broker.p &

五.生成namesrv默认配置

sh mqnamesrv -p >namesrv.p
可以通过 sh mqnamesrv -h 查看命令参数。

启动命令
nohup sh mqnamesrv -c namesrv.p &

六操作系统配置
os.sh大概意思是根据rocketmq的特点，修改系统参数，修改磁盘调度算法。
接下来我们操作一下，这个如要root权限

vim /etc/sysctl.conf增加

vm.overcommit_memory=1
vm.min_free_kbytes=5000000
vm.drop_caches=1
vm.zone_reclaim_mode=0
vm.max_map_count=655360
vm.dirty_background_ratio=50
vm.dirty_ratio=50
vm.page-cluster=3
vm.dirty_writeback_centisecs=360000
vm.swappiness=10

修改完成后sysctl -p

修改最大打开文件描述数

vim /etc/security/limits.conf添加
* soft nofile 655350
* hard nofile 655350

退出当前用户重新login就会生效,使用ulimit -n验证下。

修改io调试算法为deadline

查看当前系统支持的IO调度算法
[rocketmq@vtfsdb3 bin]$ dmesg | grep -i scheduler

查看当前系统默认io调度算法
[rocketmq@vtfsdb3 bin]$ cat /sys/block/sda/queue/scheduler
noop anticipatory deadline [cfq]

注意中间的sda换成实际运行存储rocketmq的磁盘，这个自己通过fdisk,df自己找出来吧，

临时更改I/O调度方法:
echo 'deadline' > /sys/block/sda/queue/scheduler

将这句加入开机启动/etc/rc.local中吧

修改完成，确认下
cat /sys/block/sda/queue/scheduler

jvm参数调整

cd alibaba-rocketmq/bin

有两个文件runserver.sh和runbroker.sh两个shell脚本，分别是name server和broker server的配置启动脚本
#vim修改runbroker.sh、runserver.sh两个文件
-------------------------------------
JAVA_OPT="${JAVA_OPT} -server -Xms512m -Xmx512m -Xmn256m -XX:PermSize=128m -XX:MaxPermSize=320m"
###Xms启动时内存，Xmx最大内存，Xmn最小内存

测试代码：
生产者：
/**
* Copyright (C) 2010-2013 Alibaba Group Holding Limited
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
*     http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.alibaba.rocketmq.example.quickstart;

import com.alibaba.rocketmq.client.exception.MQClientException;
import com.alibaba.rocketmq.client.producer.DefaultMQProducer;
import com.alibaba.rocketmq.client.producer.SendResult;
import com.alibaba.rocketmq.common.message.Message;

/**
* Producer，发送消息
*
*/
public class Producer {
    public static void main(String[] args) throws MQClientException, InterruptedException {
       //DefaultCluster please_rename_unique_group_name
    DefaultMQProducer producer = new DefaultMQProducer("ProducerGroupName");
    // producer.setNamesrvAddr("");//
    producer.setNamesrvAddr("192.168.6.57:9876");
    /// producer.setInstanceName("");
    producer.setInstanceName("Product");
    producer.start();

        for (int i = 0; i < 1000; i++) {
            try {
                Message msg = new Message("dell-PC",// topic   TopicTest
                    "TagA",// tag
                    ("Hello RocketMQ 测试信息------" + i).getBytes()// body
                        );
                SendResult sendResult = producer.send(msg);
                System.out.println(i+"--"+sendResult);
                Thread.sleep(30);

            }
            catch (Exception e) {
                e.printStackTrace();
                Thread.sleep(1000);
            }
        }

        producer.shutdown();
    }
}

消费者：
/**
* Copyright (C) 2010-2013 Alibaba Group Holding Limited
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
*     http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.alibaba.rocketmq.example.quickstart;

import java.util.List;

import com.alibaba.rocketmq.client.consumer.DefaultMQPushConsumer;
import com.alibaba.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext;
import com.alibaba.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import com.alibaba.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import com.alibaba.rocketmq.client.exception.MQClientException;
import com.alibaba.rocketmq.common.consumer.ConsumeFromWhere;
import com.alibaba.rocketmq.common.message.MessageExt;

/**
* Consumer，订阅消息
*/
public class Consumer {

    public static void main(String[] args) throws InterruptedException, MQClientException {
        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer
        ("please_rename_unique_group_name_4");//("please_rename_unique_group_name_4");

        consumer.setNamesrvAddr("192.168.6.57:9876");
        consumer.setInstanceName("Consumber");
        /**
         * 设置Consumer第一次启动是从队列头部开始消费还是队列尾部开始消费<br>
         * 如果非第一次启动，那么按照上次消费的位置继续消费
         */
        consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_FIRST_OFFSET);

        consumer.subscribe("dell-PC", "*"); //TopicTest
      // consumer.subscribe("BenchmarkTest", "*");
       // consumer.subscribe("benchmark_consumer_39", "*");

        consumer.registerMessageListener(new MessageListenerConcurrently() {

            @Override
            public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs,
                    ConsumeConcurrentlyContext context) {
                System.out.println(Thread.currentThread().getName() + " Receive New Messages: " + msgs);
                for(MessageExt item:msgs){
                System.out.println(new String(item.getBody()));
                }
                return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
            }
        });

        consumer.start();

        System.out.println("Consumer Started.");
    }
}

生产者运行结果：

消费者运行结果：

七. mqadmin 命令使用：
cd /opt/mq/alibaba-rocketmq/bin

1. 查看所有消费组group:
   sh mqadmin consumerProgress -n 192.168.6.57:9876
2. 查看指定消费组下的所有topic数据堆积情况：
    sh mqadmin consumerProgress -n 192.168.6.57:9876 -g benchmark_consumer_33
3. 查看所有topic :
     sh mqadmin topicList -n 192.168.6.57:9876
4. 查看topic信息列表详情统计
   sh mqadmin topicstatus -n 192.168.6.57:9876 -t myTopicTest1
5. 新增topic
   sh mqadmin updateTopic –n 192.168.6.57 –c groupname –t topicname
6. 删除topic
sh mqadmin deleteTopic –n 192.168.6.57:9876 –c groupname –t topicname

手动来配置mq参数：

#所属集群名字
brokerClusterName=rocketmq-cluster
#broker名字，注意此处不同的配置文件填写的不一样
brokerName=broker-a|broker-b
#0 表示 Master，>0 表示 Slave
brokerId=0
#nameServer地址，分号分割
namesrvAddr=192.168.1.101:9876;192.168.1.102:9876
#在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic，建议线下开启，线上关闭
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组，建议线下开启，线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口
listenPort=10911
#删除文件时间点，默认凌晨 4点
deleteWhen=04
#文件保留时间，默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条，根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/usr/local/alibaba-rocketmq/store
#commitLog 存储路径
storePathCommitLog=/usr/local/alibaba-rocketmq/store/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/usr/local/alibaba-rocketmq/store/consumequeue
#消息索引存储路径
storePathIndex=/usr/local/alibaba-rocketmq/store/index
#checkpoint 文件存储路径
storeCheckpoint=/usr/local/alibaba-rocketmq/store/checkpoint
#abort 文件存储路径
abortFile=/usr/local/alibaba-rocketmq/store/abort
#限制的消息大小
maxMessageSize=65536

#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=ASYNC_MASTER

#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128