`
maosheng
  • 浏览: 565231 次
  • 性别: Icon_minigender_1
  • 来自: 北京
社区版块
存档分类
最新评论

Linux 下 Redis Cluster 搭建

阅读更多
Redis集群演变过程:

单机版----->主从复制------>哨兵(小公司)----->集群(大公司)

Redis cluster集群:

(1)所有的redis节点彼此互联(PING-PONG机制),内部使用二进制协议优化传输速度和带宽.

(2)节点的fail是通过集群中超过半数的节点检测失效时才生效.通过投票机制

(3)客户端与redis节点直连,不需要中间proxy层.客户端不需要连接集群所有节点,连接集群中任何一个可用节点即可

(4)redis-cluster把所有的物理节点映射到[0-16383]slot上,cluster 负责维护node<->slot<->value

(5)redis 集群中内置了 16384 个哈希槽,当需要在 Redis 集群中放置一个 key-value 时,redis 先对 key 使用 crc16 算法算出一个结果,然后把结果对 16384 求余数,这样每个 key 都会对应一个编号在 0-16383 之间的哈希槽,redis 会根据节点数量大致均等的将哈希槽映射到不同的节点

由于Redis的容错投票机制是集群中过半数的节点认为某个节点检测失效时才生效,所以搭建最小集群模式至少需要三个节点,但是为了集群的高可用,就需要为每一个节点增加一个备份机,所以就需要六台服务器。

多个主节点、每个主节点可附带多个从节点、多个主节点共分16384(0---16383)个槽位(slot)


Redis cluster分片特点:

1)无中心架构,支持动态扩容
2)具备Sentinel的监控和自动切换能力
3)客户端不需要连接集群所有节点,连接集群中任何一个可用节点即可
4)高性能,客户端直连redis服务,不需要使用代理

Redis cluster有固定16384个slot,每次写入,会对key计算CRC16值,然后对16384求模,计算出应该存放slot位置,然后会找到对应的master节点。

假设有三主三从,那么slot在master划分:[0-5460],[5461-10922], [10923-16383],每个key都会找到属于自己的master节点,在master节点中进行主从复制

集群中某个master节点宕机,会进行主备切换,如果主从全部宕机,由于分片,只会影响相邻节点,引发数据迁移,迁移代价较小,而其它节点正常工作,不会因为缓存大面积失效造成缓存雪崩,造成数据库压力过大甚至宕机。

选举策略:每个从节点,都根据自己对 master 复制数据的 offset,来设置一个选举时间,offset 越大(复制数据越多)的从节点,选举时间越靠前,优先进行选举,半数以上投票通过,该从节点可以切换成为master。


5.0.9 版本搭建:

主节点M1,从节点S2 192.101.11.153:7001,192.101.11.153:7002
主节点M2,从节点S3 192.101.11.154:7003,192.101.11.154:7004
主节点M3,从节点S1 192.101.11.155:7005,192.101.11.155:7006


M1:192.101.11.153:7001 S1:192.101.11.155:7006
M2:192.101.11.154:7003 S2:192.101.11.153:7002
M3:192.101.11.155:7005 S3:192.101.11.154:7004


一、下载安装包

http://download.redis.io/releases/
http://download.redis.io/releases/redis-5.0.9.tar.gz

创建目录:
mkdir -p /opt/redis-cluster

提示:redis-5.0.9.tar.gz 上传到三个服务器该目录:/opt/redis-cluster

二、解压编译

安装编译环境:

yum install -y gcc gcc-c++ make
yum -y update

创建目录:

mkdir -p /usr/local/redis-cluster/

解压安装包:
cd /opt/redis-cluster

tar -zxvf redis-5.0.5.tar.gz

mv /opt/redis-cluster/redis-5.0.9 /usr/local/redis-cluster/

编译安装:
cd /usr/local/redis-cluster/redis-5.0.9/

make PREFIX=/usr/local/redis install

检测:

make test

三、集群分配

主节点M1,从节点S2 192.101.11.153:
mkdir -p /usr/local/redis-cluster/redis-m1
mkdir -p /usr/local/redis-cluster/redis-s2
cp /usr/local/redis/bin /usr/local/redis-cluster/redis-m1 -r
cp /usr/local/redis-cluster/redis-5.0.9/redis.conf  /usr/local/redis-cluster/redis-m1
cp /usr/local/redis/bin /usr/local/redis-cluster/redis-s2 -r
cp /usr/local/redis-cluster/redis-5.0.9/redis.conf  /usr/local/redis-cluster/redis-s2

主节点M2,从节点S3 192.101.11.154:
mkdir -p /usr/local/redis-cluster/redis-m2
mkdir -p /usr/local/redis-cluster/redis-s3
cp /usr/local/redis/bin /usr/local/redis-cluster/redis-m2 -r
cp /usr/local/redis-cluster/redis-5.0.9/redis.conf  /usr/local/redis-cluster/redis-m2
cp /usr/local/redis/bin /usr/local/redis-cluster/redis-s3 -r
cp /usr/local/redis-cluster/redis-5.0.9/redis.conf  /usr/local/redis-cluster/redis-s3

主节点M3,从节点S1 192.101.11.155:
mkdir -p /usr/local/redis-cluster/redis-m3
mkdir -p /usr/local/redis-cluster/redis-s1
cp /usr/local/redis/bin /usr/local/redis-cluster/redis-m3 -r
cp /usr/local/redis-cluster/redis-5.0.9/redis.conf  /usr/local/redis-cluster/redis-m3
cp /usr/local/redis/bin /usr/local/redis-cluster/redis-s1 -r
cp /usr/local/redis-cluster/redis-5.0.9/redis.conf  /usr/local/redis-cluster/redis-s1


四、配置文件

主节点M1 192.101.11.153:
mkdir -p /data/redis/m1/logs/
vi /usr/local/redis-cluster/redis-m1/redis.conf

bind 192.101.11.153  ##绑定IP,把 bind 127.0.0.1 给注释掉,这里的bind指的是只有指定的网段才能远程访问这个redis,注释掉后,就没有这个限制了,监听所有地址
port 7001 #端口
cluster-enabled yes #启用集群模式
cluster-node-timeout 5000 #超时时间5s
appendonly yes # 开启AOF日志持久化
daemonize yes #后台运行
cluster-config-file /usr/local/redis-cluster/redis-m1/nodes-7001.conf # redis群集名称文件设置
pidfile /var/run/redis_7001.pid
dbfilename 7001-dump.rdb
dir /data/redis/m1
logfile /data/redis/m1/logs/redis-7001.log


从节点S2 192.101.11.153:
mkdir -p /data/redis/s2/logs/
vi /usr/local/redis-cluster/redis-s2/redis.conf

bind 192.101.11.153  ##绑定IP,把 bind 127.0.0.1 给注释掉,这里的bind指的是只有指定的网段才能远程访问这个redis,注释掉后,就没有这个限制了,监听所有地址
port 7002 #端口
cluster-enabled yes #启用集群模式
cluster-node-timeout 5000 #超时时间5s
appendonly yes # 开启AOF日志持久化
daemonize yes #后台运行
cluster-config-file /usr/local/redis-cluster/redis-s2/nodes-7002.conf  # redis群集名称文件设置
pidfile /var/run/redis_7002.pid
dbfilename 7002-dump.rdb
dir /data/redis/s2
logfile /data/redis/s2/logs/redis-7002.log


主节点M2 192.101.11.154:
mkdir -p /data/redis/m2/logs/
vi /usr/local/redis-cluster/redis-m2/redis.conf

bind 192.101.11.154  ##绑定IP,把 bind 127.0.0.1 给注释掉,这里的bind指的是只有指定的网段才能远程访问这个redis,注释掉后,就没有这个限制了,监听所有地址
port 7003 #端口
cluster-enabled yes #启用集群模式
cluster-node-timeout 5000 #超时时间5s
appendonly yes # 开启AOF日志持久化
daemonize yes #后台运行
cluster-config-file /usr/local/redis-cluster/redis-m2/nodes-7003.conf # redis群集名称文件设置
pidfile /var/run/redis_7003.pid
dbfilename 7003-dump.rdb
dir /data/redis/m2
logfile /data/redis/m2/logs/redis-7003.log


从节点S3 192.101.11.154:
mkdir -p /data/redis/s3/logs/
vi /usr/local/redis-cluster/redis-s3/redis.conf

bind 192.101.11.154  ##绑定IP,把 bind 127.0.0.1 给注释掉,这里的bind指的是只有指定的网段才能远程访问这个redis,注释掉后,就没有这个限制了,监听所有地址
port 7004 #端口
cluster-enabled yes #启用集群模式
cluster-node-timeout 5000 #超时时间5s
appendonly yes # 开启AOF日志持久化
daemonize yes #后台运行
cluster-config-file /usr/local/redis-cluster/redis-s3/nodes-7004.conf   # redis群集名称文件设置
pidfile /var/run/redis_7004.pid 
dbfilename 7004-dump.rdb
dir /data/redis/s3
logfile /data/redis/s3/logs/redis-7004.log


主节点M3 192.101.11.155:
mkdir -p /data/redis/m3/logs/
vi /usr/local/redis-cluster/redis-m3/redis.conf

bind 192.101.11.155  ##绑定IP,把 bind 127.0.0.1 给注释掉,这里的bind指的是只有指定的网段才能远程访问这个redis,注释掉后,就没有这个限制了,监听所有地址
port 7005 #端口
cluster-enabled yes #启用集群模式
cluster-node-timeout 5000 #超时时间5s
appendonly yes # 开启AOF日志持久化
daemonize yes #后台运行
cluster-config-file /usr/local/redis-cluster/redis-m3/nodes-7005.conf   # redis群集名称文件设置
pidfile /var/run/redis_7005.pid
dbfilename 7005-dump.rdb
dir /data/redis/m3
logfile /data/redis/m3/logs/redis-7005.log


从节点S1 192.101.11.155:
mkdir -p /data/redis/s1/logs/
vi /usr/local/redis-cluster/redis-s1/redis.conf

bind 192.101.11.155  ##绑定IP,把 bind 127.0.0.1 给注释掉,这里的bind指的是只有指定的网段才能远程访问这个redis,注释掉后,就没有这个限制了,监听所有地址
port 7006 #端口
cluster-enabled yes #启用集群模式
cluster-node-timeout 5000 #超时时间5s
appendonly yes # 开启AOF日志持久化
daemonize yes #后台运行
cluster-config-file /usr/local/redis-cluster/redis-s1/nodes-7006.conf   # redis群集名称文件设置
pidfile /var/run/redis_7006.pid
dbfilename 7006-dump.rdb
dir /data/redis/s1
logfile /data/redis/s1/logs/redis-7006.log

其他可选设置:
1)开启远程访问:protected-mode no
2)设置redis集群密码:masterauth 123456 和 requirepass 123456     ##密码是123456 
   注意:所有节点的密码都必须一致,masterauth也要加

五、启动节点

配置好了之后,就可以启动节点了,三台机器分别启动。使用redis-server命令启动,启动时注意指定配置文件

主节点M1 192.101.11.153:
/usr/local/redis-cluster/redis-m1/bin/redis-server /usr/local/redis-cluster/redis-m1/redis.conf
从节点S2 192.101.11.153:
/usr/local/redis-cluster/redis-s2/bin/redis-server /usr/local/redis-cluster/redis-s2/redis.conf

[root@hadoop001 redis-s2]# ps -ef|grep redis
root      2609     1  0 00:43 ?        00:00:00 /usr/local/redis-cluster/redis-m1/bin/redis-server 192.101.11.153:7001 [cluster]
root      2660     1  0 00:44 ?        00:00:00 /usr/local/redis-cluster/redis-s2/bin/redis-server 192.101.11.153:7002 [cluster]


主节点M2 192.101.11.154:
/usr/local/redis-cluster/redis-m2/bin/redis-server /usr/local/redis-cluster/redis-m2/redis.conf
从节点S3 192.101.11.154:
/usr/local/redis-cluster/redis-s3/bin/redis-server /usr/local/redis-cluster/redis-s3/redis.conf

[root@hadoop002 redis-m2]# ps -ef|grep redis
root     21229     1  0 00:47 ?        00:00:00 /usr/local/redis-cluster/redis-m2/bin/redis-server 192.101.11.154:7003 [cluster]
root     21234     1  0 00:48 ?        00:00:00 /usr/local/redis-cluster/redis-s3/bin/redis-server 192.101.11.154:7004 [cluster]


主节点M3 192.101.11.155:
/usr/local/redis-cluster/redis-m3/bin/redis-server /usr/local/redis-cluster/redis-m3/redis.conf
从节点S1 192.101.11.155:
/usr/local/redis-cluster/redis-s1/bin/redis-server /usr/local/redis-cluster/redis-s1/redis.conf

[root@hadoop003 redis-m3]# ps -ef|grep redis
root      2239     1  0 00:51 ?        00:00:00 /usr/local/redis-cluster/redis-m3/bin/redis-server 192.101.11.155:7005 [cluster]
root      2261     1  0 00:51 ?        00:00:00 /usr/local/redis-cluster/redis-s1/bin/redis-server 192.101.11.155:7006 [cluster]


六、创建集群

三个节点都启动成功后,就可以创建集群了。Redis官方提供了命令,我们敲一下就可以了,非常方便。

提示:在其中一台服务器执行即可,选择主节点M1(192.101.11.153)

./redis-cli --cluster create ip1:port1 ip2:port2 ip3:port3

cd /usr/local/redis/bin

./redis-cli --cluster create 192.101.11.153:7001 192.101.11.154:7003 192.101.11.155:7005 192.101.11.153:7002 192.101.11.154:7004 192.101.11.155:7006 --cluster-replicas 1

提示:--cluster-replicas 1 命令的意思:一主一从配置,六个节点就是:三主三从

./redis-cli --cluster create 192.101.11.153:7001 192.101.11.154:7003 192.101.11.155:7005 192.101.11.153:7002 192.101.11.154:7004 192.101.11.155:7006 --cluster-replicas 1 -a 123456

提示:如果我们设置了redis集群的密码,那么就要带上 -a 123456 (密码),其他同理

>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.101.11.154:7004 to 192.101.11.153:7001
Adding replica 192.101.11.155:7006 to 192.101.11.154:7003
Adding replica 192.101.11.153:7002 to 192.101.11.155:7005
M: e69af493737ff8fe6ced6a9edd41814c79fe4c54 192.101.11.153:7001
   slots:[0-5460] (5461 slots) master
M: 46683ba127a4c7ca9ecb5d56316b925a575d1741 192.101.11.154:7003
   slots:[5461-10922] (5462 slots) master
M: ff08cdb6ab20ec6649639a28481f3dcacdb83e6d 192.101.11.155:7005
   slots:[10923-16383] (5461 slots) master
S: af1887d5eef14a65793fd19fb0f9a24fb8c43f3c 192.101.11.153:7002
   replicates ff08cdb6ab20ec6649639a28481f3dcacdb83e6d
S: d3964275adfba68de04f9b75ece8a42b94207ed6 192.101.11.154:7004
   replicates e69af493737ff8fe6ced6a9edd41814c79fe4c54
S: 8971bd6b6071b6bcd16a9b03e71217d01555e3ab 192.101.11.155:7006
   replicates 46683ba127a4c7ca9ecb5d56316b925a575d1741
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
...
>>> Performing Cluster Check (using node 192.101.11.153:7001)
M: e69af493737ff8fe6ced6a9edd41814c79fe4c54 192.101.11.153:7001
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: ff08cdb6ab20ec6649639a28481f3dcacdb83e6d 192.101.11.155:7005
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 8971bd6b6071b6bcd16a9b03e71217d01555e3ab 192.101.11.155:7006
   slots: (0 slots) slave
   replicates 46683ba127a4c7ca9ecb5d56316b925a575d1741
S: af1887d5eef14a65793fd19fb0f9a24fb8c43f3c 192.101.11.153:7002
   slots: (0 slots) slave
   replicates ff08cdb6ab20ec6649639a28481f3dcacdb83e6d
S: d3964275adfba68de04f9b75ece8a42b94207ed6 192.101.11.154:7004
   slots: (0 slots) slave
   replicates e69af493737ff8fe6ced6a9edd41814c79fe4c54
M: 46683ba127a4c7ca9ecb5d56316b925a575d1741 192.101.11.154:7003
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.


七、测试

查看帮助命令:

./redis-cli --help

检查集群状态:

./redis-cli --cluster check 192.101.11.153:7001

./redis-cli --cluster check 192.101.11.153:7001 -a 123456

提示:如果我们设置了redis集群的密码,那么就要带上 -a 123456 (密码),其他同理


[root@hadoop001 bin]# ./redis-cli --cluster check 192.101.11.153:7001
192.101.11.153:7001 (e69af493...) -> 0 keys | 5461 slots | 1 slaves.
192.101.11.155:7005 (ff08cdb6...) -> 0 keys | 5461 slots | 1 slaves.
192.101.11.154:7003 (46683ba1...) -> 0 keys | 5462 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.101.11.153:7001)
M: e69af493737ff8fe6ced6a9edd41814c79fe4c54 192.101.11.153:7001
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: ff08cdb6ab20ec6649639a28481f3dcacdb83e6d 192.101.11.155:7005
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 8971bd6b6071b6bcd16a9b03e71217d01555e3ab 192.101.11.155:7006
   slots: (0 slots) slave
   replicates 46683ba127a4c7ca9ecb5d56316b925a575d1741
S: af1887d5eef14a65793fd19fb0f9a24fb8c43f3c 192.101.11.153:7002
   slots: (0 slots) slave
   replicates ff08cdb6ab20ec6649639a28481f3dcacdb83e6d
S: d3964275adfba68de04f9b75ece8a42b94207ed6 192.101.11.154:7004
   slots: (0 slots) slave
   replicates e69af493737ff8fe6ced6a9edd41814c79fe4c54
M: 46683ba127a4c7ca9ecb5d56316b925a575d1741 192.101.11.154:7003
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.


./redis-cli -c -h 192.101.11.153 -p 7001

./redis-cli -c -h 192.101.11.153 -p 7001 -a 123456

提示:如果我们设置了redis集群的密码,那么就要带上 -a 123456 (密码),其他同理

>cluster info

[root@hadoop001 bin]# ./redis-cli -c -h 192.101.11.153 -p 7001
192.101.11.153:7001> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:352
cluster_stats_messages_pong_sent:321
cluster_stats_messages_sent:673
cluster_stats_messages_ping_received:316
cluster_stats_messages_pong_received:352
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:673

192.101.11.153:7001> set hi redis
-> Redirected to slot [16140] located at 192.101.11.155:7005
OK

192.101.11.155:7005> get hi
"redis"

测试性能:

redis-benchmark -h 192.101.11.153 -p 7001 -c 100 -n 100000     ##100个并发连接,100000个请求,检测host为localhost 端口为7002的redis服务器性能


查看复制信息:
./redis-cli -c -h 192.101.11.153 -p 7001 -a 123456 info replication


八、手动添加slave方式

以上创建集群方式是自动分配的 master 和 slave 的组合关系,而有时我们想按照规划好的组合关系,手动匹配他们的对应关系。

创建集群(创建不含slaver 的集群):
./redis-cli --cluster create 192.101.11.153:7001 192.101.11.154:7003 192.101.11.155:7005 --cluster-replicas 0

提示:--cluster-replicas 0 命令的意思:一主零从配置,三个节点就是:三主零从

./redis-cli --cluster create 192.101.11.153:7001 192.101.11.154:7003 192.101.11.155:7005 --cluster-replicas 0 -a 123456
提示:如果我们设置了redis集群的密码,那么就要带上 -a 123456 (密码),其他同理

[root@hadoop001 bin]# ./redis-cli --cluster create 192.101.11.153:7001 192.101.11.154:7003 192.101.11.155:7005 --cluster-replicas 0
>>> Performing hash slots allocation on 3 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
M: 0182025e5d7a7e248c0bf53bc91c1054b7f53122 192.101.11.153:7001
   slots:[0-5460] (5461 slots) master
M: ac978c3dd9ba47b1b83cbbd1ea4404735b1841b3 192.101.11.154:7003
   slots:[5461-10922] (5462 slots) master
M: e9b9eca0e106dcb12e74a71d231a3fdaf622bada 192.101.11.155:7005
   slots:[10923-16383] (5461 slots) master
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 192.101.11.153:7001)
M: 0182025e5d7a7e248c0bf53bc91c1054b7f53122 192.101.11.153:7001
   slots:[0-5460] (5461 slots) master
M: ac978c3dd9ba47b1b83cbbd1ea4404735b1841b3 192.101.11.154:7003
   slots:[5461-10922] (5462 slots) master
M: e9b9eca0e106dcb12e74a71d231a3fdaf622bada 192.101.11.155:7005
   slots:[10923-16383] (5461 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.


查看集群运行状态:
[root@hadoop001 bin]# ./redis-cli --cluster check 192.101.11.153:7001
192.101.11.153:7001 (0182025e...) -> 0 keys | 5461 slots | 0 slaves.
192.101.11.154:7003 (ac978c3d...) -> 0 keys | 5462 slots | 0 slaves.
192.101.11.155:7005 (e9b9eca0...) -> 0 keys | 5461 slots | 0 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.101.11.153:7001)
M: 0182025e5d7a7e248c0bf53bc91c1054b7f53122 192.101.11.153:7001
   slots:[0-5460] (5461 slots) master
M: ac978c3dd9ba47b1b83cbbd1ea4404735b1841b3 192.101.11.154:7003
   slots:[5461-10922] (5462 slots) master
M: e9b9eca0e106dcb12e74a71d231a3fdaf622bada 192.101.11.155:7005
   slots:[10923-16383] (5461 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.


查看集群运行状态(如果是通过--cluster-replicas 1 则创建了集群的slave,并自动进行分组匹配):

./redis-cli --cluster check 192.101.11.153:7001
[root@hadoop001 bin]# ./redis-cli --cluster check 192.101.11.153:7001
192.101.11.153:7001 (e69af493...) -> 0 keys | 5461 slots | 1 slaves.
192.101.11.155:7005 (ff08cdb6...) -> 1 keys | 5461 slots | 1 slaves.
192.101.11.154:7003 (46683ba1...) -> 0 keys | 5462 slots | 1 slaves.
[OK] 1 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.101.11.153:7001)
M: e69af493737ff8fe6ced6a9edd41814c79fe4c54 192.101.11.153:7001
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: ff08cdb6ab20ec6649639a28481f3dcacdb83e6d 192.101.11.155:7005
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 8971bd6b6071b6bcd16a9b03e71217d01555e3ab 192.101.11.155:7006
   slots: (0 slots) slave
   replicates 46683ba127a4c7ca9ecb5d56316b925a575d1741
S: af1887d5eef14a65793fd19fb0f9a24fb8c43f3c 192.101.11.153:7002
   slots: (0 slots) slave
   replicates ff08cdb6ab20ec6649639a28481f3dcacdb83e6d
S: d3964275adfba68de04f9b75ece8a42b94207ed6 192.101.11.154:7004
   slots: (0 slots) slave
   replicates e69af493737ff8fe6ced6a9edd41814c79fe4c54
M: 46683ba127a4c7ca9ecb5d56316b925a575d1741 192.101.11.154:7003
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.


删除当前的slave(如果是通过--cluster-replicas 1 创建了集群的slave,则删除重新匹配对应关系):
./redis-cli -a 123456 --cluster del-node 192.101.11.155:7006 '8971bd6b6071b6bcd16a9b03e71217d01555e3ab'
./redis-cli -a 123456 --cluster del-node 192.101.11.153:7002 'af1887d5eef14a65793fd19fb0f9a24fb8c43f3c'
./redis-cli -a 123456 --cluster del-node 192.101.11.154:7004 'd3964275adfba68de04f9b75ece8a42b94207ed6'

提示:
slave1 id:8971bd6b6071b6bcd16a9b03e71217d01555e3ab
slave2 id:af1887d5eef14a65793fd19fb0f9a24fb8c43f3c
slave3 id:d3964275adfba68de04f9b75ece8a42b94207ed6


[root@hadoop001 bin]# ./redis-cli --cluster del-node 192.101.11.155:7006 '8971bd6b6071b6bcd16a9b03e71217d01555e3ab'
>>> Removing node 8971bd6b6071b6bcd16a9b03e71217d01555e3ab from cluster 192.101.11.155:7006
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.

[root@hadoop001 bin]# ./redis-cli --cluster del-node 192.101.11.154:7004 'd3964275adfba68de04f9b75ece8a42b94207ed6'
>>> Removing node d3964275adfba68de04f9b75ece8a42b94207ed6 from cluster 192.101.11.154:7004
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.

[root@hadoop001 bin]# ./redis-cli --cluster del-node 192.101.11.153:7002 'af1887d5eef14a65793fd19fb0f9a24fb8c43f3c'
>>> Removing node af1887d5eef14a65793fd19fb0f9a24fb8c43f3c from cluster 192.101.11.153:7002
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.

查看集群当前状态:
[root@hadoop001 bin]# ./redis-cli --cluster check 192.101.11.153:7001
192.101.11.153:7001 (e69af493...) -> 0 keys | 5461 slots | 0 slaves.
192.101.11.155:7005 (ff08cdb6...) -> 1 keys | 5461 slots | 0 slaves.
192.101.11.154:7003 (46683ba1...) -> 0 keys | 5462 slots | 0 slaves.
[OK] 1 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.101.11.153:7001)
M: e69af493737ff8fe6ced6a9edd41814c79fe4c54 192.101.11.153:7001
   slots:[0-5460] (5461 slots) master
M: ff08cdb6ab20ec6649639a28481f3dcacdb83e6d 192.101.11.155:7005
   slots:[10923-16383] (5461 slots) master
M: 46683ba127a4c7ca9ecb5d56316b925a575d1741 192.101.11.154:7003
   slots:[5461-10922] (5462 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

启动salve节点Redis:
从节点S1 192.101.11.155:
/usr/local/redis-cluster/redis-s1/bin/redis-server /usr/local/redis-cluster/redis-s1/redis.conf

从节点S2 192.101.11.153:
/usr/local/redis-cluster/redis-s2/bin/redis-server /usr/local/redis-cluster/redis-s2/redis.conf

从节点S3 192.101.11.154:
/usr/local/redis-cluster/redis-s3/bin/redis-server /usr/local/redis-cluster/redis-s3/redis.conf


集群主从对应关系:
M1:192.101.11.153:7001 S1:192.101.11.155:7006
M2:192.101.11.154:7003 S2:192.101.11.153:7002
M3:192.101.11.155:7005 S3:192.101.11.154:7004

./redis-cli --cluster add-node 192.101.11.155:7006 192.101.11.153:7001 --cluster-slave --cluster-master-id e69af493737ff8fe6ced6a9edd41814c79fe4c54
或者
./redis-cli -a 123456 --cluster add-node 192.101.11.155:7006 192.101.11.153:7001 --cluster-slave --cluster-master-id e69af493737ff8fe6ced6a9edd41814c79fe4c54

./redis-cli --cluster add-node 192.101.11.153:7002 192.101.11.153:7001 --cluster-slave --cluster-master-id 46683ba127a4c7ca9ecb5d56316b925a575d1741
或者
./redis-cli -a 123456 --cluster add-node 192.101.11.153:7002 192.101.11.153:7001 --cluster-slave --cluster-master-id 46683ba127a4c7ca9ecb5d56316b925a575d1741

./redis-cli --cluster add-node 192.101.11.154:7004 192.101.11.153:7001 --cluster-slave --cluster-master-id ff08cdb6ab20ec6649639a28481f3dcacdb83e6d
或者
./redis-cli -a 123456 --cluster add-node 192.101.11.154:7004 192.101.11.153:7001 --cluster-slave --cluster-master-id ff08cdb6ab20ec6649639a28481f3dcacdb83e6d

提示:
192.101.11.155:7006 为slave node, master1 id: e69af493737ff8fe6ced6a9edd41814c79fe4c54
192.101.11.153:7002 为slave node, master2 id: 46683ba127a4c7ca9ecb5d56316b925a575d1741
192.101.11.154:7004 为slave node, master3 id: ff08cdb6ab20ec6649639a28481f3dcacdb83e6d

提示:这一步出现如下的报错问题:解决见最后(九、问题及解决)
[root@hadoop001 bin]# ./redis-cli --cluster add-node 192.101.11.155:7006 192.101.11.153:7001 --cluster-slave --cluster-master-id e69af493737ff8fe6ced6a9edd41814c79fe4c54
>>> Adding node 192.101.11.155:7006 to cluster 192.101.11.153:7001
>>> Performing Cluster Check (using node 192.101.11.153:7001)
M: e69af493737ff8fe6ced6a9edd41814c79fe4c54 192.101.11.153:7001
   slots:[0-5460] (5461 slots) master
M: ff08cdb6ab20ec6649639a28481f3dcacdb83e6d 192.101.11.155:7005
   slots:[10923-16383] (5461 slots) master
M: 46683ba127a4c7ca9ecb5d56316b925a575d1741 192.101.11.154:7003
   slots:[5461-10922] (5462 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[ERR] Node 192.101.11.155:7006 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.


[root@hadoop001 bin]# ./redis-cli --cluster add-node 192.101.11.155:7006 192.101.11.153:7001 --cluster-slave --cluster-master-id 0182025e5d7a7e248c0bf53bc91c1054b7f53122
>>> Adding node 192.101.11.155:7006 to cluster 192.101.11.153:7001
>>> Performing Cluster Check (using node 192.101.11.153:7001)
M: 0182025e5d7a7e248c0bf53bc91c1054b7f53122 192.101.11.153:7001
   slots:[0-5460] (5461 slots) master
M: ac978c3dd9ba47b1b83cbbd1ea4404735b1841b3 192.101.11.154:7003
   slots:[5461-10922] (5462 slots) master
M: e9b9eca0e106dcb12e74a71d231a3fdaf622bada 192.101.11.155:7005
   slots:[10923-16383] (5461 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.101.11.155:7006 to make it join the cluster.
Waiting for the cluster to join
..
>>> Configure node as replica of 192.101.11.153:7001.
[OK] New node added correctly.
You have mail in /var/spool/mail/root

[root@hadoop001 bin]# ./redis-cli --cluster add-node 192.101.11.153:7002 192.101.11.153:7001 --cluster-slave --cluster-master-id ac978c3dd9ba47b1b83cbbd1ea4404735b1841b3
>>> Adding node 192.101.11.153:7002 to cluster 192.101.11.153:7001
>>> Performing Cluster Check (using node 192.101.11.153:7001)
M: 0182025e5d7a7e248c0bf53bc91c1054b7f53122 192.101.11.153:7001
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: ac978c3dd9ba47b1b83cbbd1ea4404735b1841b3 192.101.11.154:7003
   slots:[5461-10922] (5462 slots) master
S: 9015969881858f7952263ba4ede6cb4e4b70715f 192.101.11.155:7006
   slots: (0 slots) slave
   replicates 0182025e5d7a7e248c0bf53bc91c1054b7f53122
M: e9b9eca0e106dcb12e74a71d231a3fdaf622bada 192.101.11.155:7005
   slots:[10923-16383] (5461 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.101.11.153:7002 to make it join the cluster.
Waiting for the cluster to join
...
>>> Configure node as replica of 192.101.11.154:7003.
[OK] New node added correctly.

[root@hadoop001 bin]# ./redis-cli --cluster add-node 192.101.11.154:7004 192.101.11.153:7001 --cluster-slave --cluster-master-id e9b9eca0e106dcb12e74a71d231a3fdaf622bada
>>> Adding node 192.101.11.154:7004 to cluster 192.101.11.153:7001
>>> Performing Cluster Check (using node 192.101.11.153:7001)
M: 0182025e5d7a7e248c0bf53bc91c1054b7f53122 192.101.11.153:7001
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 279dfba90e40732c6b9bf9fdbb55ba6ab54a0028 192.101.11.153:7002
   slots: (0 slots) slave
   replicates ac978c3dd9ba47b1b83cbbd1ea4404735b1841b3
M: ac978c3dd9ba47b1b83cbbd1ea4404735b1841b3 192.101.11.154:7003
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
M: e9b9eca0e106dcb12e74a71d231a3fdaf622bada 192.101.11.155:7005
   slots:[10923-16383] (5461 slots) master
S: 9015969881858f7952263ba4ede6cb4e4b70715f 192.101.11.155:7006
   slots: (0 slots) slave
   replicates 0182025e5d7a7e248c0bf53bc91c1054b7f53122
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.101.11.154:7004 to make it join the cluster.
Waiting for the cluster to join

>>> Configure node as replica of 192.101.11.155:7005.
[OK] New node added correctly.


[root@hadoop001 bin]#  ./redis-cli --cluster check 192.101.11.153:7001
192.101.11.153:7001 (0182025e...) -> 0 keys | 5461 slots | 1 slaves.
192.101.11.154:7003 (ac978c3d...) -> 0 keys | 5462 slots | 1 slaves.
192.101.11.155:7005 (e9b9eca0...) -> 0 keys | 5461 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.101.11.153:7001)
M: 0182025e5d7a7e248c0bf53bc91c1054b7f53122 192.101.11.153:7001
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 279dfba90e40732c6b9bf9fdbb55ba6ab54a0028 192.101.11.153:7002
   slots: (0 slots) slave
   replicates ac978c3dd9ba47b1b83cbbd1ea4404735b1841b3
M: ac978c3dd9ba47b1b83cbbd1ea4404735b1841b3 192.101.11.154:7003
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 8c143cba38d0978d6202785502925062f4bac6b8 192.101.11.154:7004
   slots: (0 slots) slave
   replicates e9b9eca0e106dcb12e74a71d231a3fdaf622bada
M: e9b9eca0e106dcb12e74a71d231a3fdaf622bada 192.101.11.155:7005
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 9015969881858f7952263ba4ede6cb4e4b70715f 192.101.11.155:7006
   slots: (0 slots) slave
   replicates 0182025e5d7a7e248c0bf53bc91c1054b7f53122
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.


[root@hadoop001 bin]# ./redis-cli -c -h 192.101.11.155 -p 7005 cluster nodes
e9b9eca0e106dcb12e74a71d231a3fdaf622bada 192.101.11.155:7005@17005 myself,master - 0 1597759574000 3 connected 10923-16383
8c143cba38d0978d6202785502925062f4bac6b8 192.101.11.154:7004@17004 slave e9b9eca0e106dcb12e74a71d231a3fdaf622bada 0 1597759580000 3 connected
279dfba90e40732c6b9bf9fdbb55ba6ab54a0028 192.101.11.153:7002@17002 slave ac978c3dd9ba47b1b83cbbd1ea4404735b1841b3 0 1597759577000 2 connected
0182025e5d7a7e248c0bf53bc91c1054b7f53122 192.101.11.153:7001@17001 master - 0 1597759578000 1 connected 0-5460
ac978c3dd9ba47b1b83cbbd1ea4404735b1841b3 192.101.11.154:7003@17003 master - 0 1597759580856 2 connected 5461-10922
9015969881858f7952263ba4ede6cb4e4b70715f 192.101.11.155:7006@17006 slave 0182025e5d7a7e248c0bf53bc91c1054b7f53122 0 1597759579000 1 connected

 
九、问题及解决:

[root@hadoop001 bin]# ./redis-cli --cluster add-node 192.101.11.155:7006 192.101.11.153:7001 --cluster-slave --cluster-master-id e69af493737ff8fe6ced6a9edd41814c79fe4c54
>>> Adding node 192.101.11.155:7006 to cluster 192.101.11.153:7001
>>> Performing Cluster Check (using node 192.101.11.153:7001)
M: e69af493737ff8fe6ced6a9edd41814c79fe4c54 192.101.11.153:7001
   slots:[0-5460] (5461 slots) master
M: ff08cdb6ab20ec6649639a28481f3dcacdb83e6d 192.101.11.155:7005
   slots:[10923-16383] (5461 slots) master
M: 46683ba127a4c7ca9ecb5d56316b925a575d1741 192.101.11.154:7003
   slots:[5461-10922] (5462 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[ERR] Node 192.101.11.155:7006 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.

解决方法:

1)、将需要新增的节点下aof、rdb等本地备份文件删除;

2)、同时将新Node的集群配置文件删除,即:删除你redis.conf里面cluster-config-file所在的文件;

3)、再次添加新节点如果还是报错,则登录新Node,./redis-cli -c –h –p对数据库进行清除:

172.168.63.201:7001>  flushdb      #清空当前数据库

查看集群节点:
redis-cli -p 7000 -h 192.168.1.147 cluster nodes

连接到每个master节点上进行数据删除:

./redis-cli -c -h 192.101.11.153 -p 7001

./redis-cli -c -h 192.101.11.155 -p 7005

./redis-cli -c -h 192.101.11.154 -p 7003

> flushall ##清除所有redis中的数据

[root@hadoop003 bin]# ./redis-cli -c -h 192.101.11.155 -p 7005 cluster nodes
46683ba127a4c7ca9ecb5d56316b925a575d1741 192.101.11.154:7003@17003 master - 0 15                            97755991713 2 connected 5461-10922
ff08cdb6ab20ec6649639a28481f3dcacdb83e6d 192.101.11.155:7005@17005 myself,master                             - 0 1597755990000 3 connected 10923-16383
e69af493737ff8fe6ced6a9edd41814c79fe4c54 192.101.11.153:7001@17001 master - 0 15                            97755990711 1 connected 0-5460
[root@hadoop003 bin]# ./redis-cli -c -h 192.101.11.155 -p 7005
192.101.11.155:7005> flushall
OK
192.101.11.155:7005>exit
[root@hadoop003 bin]# ./redis-cli -c -h 192.101.11.153 -p 7001
192.101.11.153:7001> flushall
OK
192.101.11.153:7001> exit
[root@hadoop003 bin]# ./redis-cli -c -h 192.101.11.154 -p 7003
192.101.11.154:7003> flushall
OK
192.101.11.154:7003> exit

查看集群当前状态:
./redis-cli --cluster check 192.101.11.153:7001
192.101.11.153:7001 (e69af493...) -> 0 keys | 5461 slots | 0 slaves.
192.101.11.155:7005 (ff08cdb6...) -> 0 keys | 5461 slots | 0 slaves.
192.101.11.154:7003 (46683ba1...) -> 0 keys | 5462 slots | 0 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.101.11.153:7001)
M: e69af493737ff8fe6ced6a9edd41814c79fe4c54 192.101.11.153:7001
   slots:[0-5460] (5461 slots) master
M: ff08cdb6ab20ec6649639a28481f3dcacdb83e6d 192.101.11.155:7005
   slots:[10923-16383] (5461 slots) master
M: 46683ba127a4c7ca9ecb5d56316b925a575d1741 192.101.11.154:7003
   slots:[5461-10922] (5462 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.












分享到:
评论

相关推荐

    Linux下Redis集群 搭建教程

    在安装完成后,需要创建一个主文件夹,名为redis-cluster,并在其下创建6个子文件夹,分别命名为7000、7001、7002、7003、7004、7005。 三、配置Redis 接下来,需要配置Redis。在每个子文件夹下面,需要创建一个...

    linux下安装redis以及搭建redis集群

    在Linux环境下安装Redis并搭建Redis集群是一个涉及到系统管理、网络配置和数据库操作的重要任务。Redis是一种高性能的键值存储系统,常用于缓存、消息队列等场景,而Redis集群则能提供高可用性和数据冗余,确保服务...

    redis cluster 4.0.8 集群配置文档

    ### Redis Cluster 4.0.8 配置与管理知识点详解 #### 一、环境准备与Redis安装 **1.1 Redis安装包获取** ...通过以上步骤,您可以成功搭建并管理基于Redis 4.0.8的集群环境,实现高效的数据处理和服务扩展能力。

    Redis集群搭建篇.docx

    ### Redis集群搭建详解 #### 一、概述 Redis(Remote Dictionary Server)是一种开源的高性能键值存储系统,常被用于数据库、缓存和消息中间件等场景。在高可用性和扩展性的需求下,Redis提供了集群功能来实现数据...

    redis cluster集群

    本文档的目的是指导读者如何在实际环境中部署和配置Redis Cluster,包括硬件选择、操作系统的选择、环境准备以及具体的集群搭建步骤,以确保用户能够顺利地构建一个稳定的Redis分布式环境。 1.2 参考文档 在部署...

    linux redis集群部署

    ### Linux Redis 集群部署详解 ...通过以上步骤,可以在Linux环境下成功搭建一个包含两个服务器、六个节点的Redis集群。这种集群模式不仅能够提供高可用性,还支持水平扩展,非常适合大规模数据处理场景下的应用。

    Redis集群搭建及维护文档.docx

    - 在/home/目录下创建名为redis-cluster的文件夹,并在其中创建6个子目录,代表6个节点,如nodes-7001至nodes-7006。 4. **配置Redis节点**: - 对每个节点的redis.conf进行配置,主要修改以下关键设置: - 将...

    自学Redis集群搭建过程--全网最详细.docx

    - 复制bin目录到`redis-cluster/redis01`目录下。 - 进入`redis01`目录,删除`appendonly.aof`和`dump.rdb`文件。 - 修改`redis.conf`文件中的端口号为7001,并启用集群模式。 4. **复制并配置其他实例**: - ...

    redis 离线集群redis5.0以上版本

    在Redis 5.0以上版本中,特别是5.0.2这个Linux版本,引入了一些重要的变化和改进,这对于搭建和管理Redis集群至关重要。 1. **Redis Cluster支持**: Redis 5.0引入了官方的集群解决方案——Redis Cluster,它允许...

    非常详细的redis集群搭建教程

    ### 知识点一:Redis集群的基本概念与优势 Redis 集群是一种高可用性的...以上步骤详细介绍了如何在CentOS 7环境下搭建一个包含6个节点的Redis集群。在整个过程中需要注意各个步骤的细节处理,以确保集群的正常运行。

    redis_cluster离线安装包及其安装手册

    在Linux环境下安装Redis Cluster时,离线安装包是针对那些无法直接连接到互联网或者网络环境受限的环境而设计的。这份"redis_cluster离线安装包及其安装手册"包含了所有必要的依赖包和软件,确保了即使在没有网络的...

    linux版redis配置文件及redis安装包.zip

    本资源包含了一个Linux版的Redis安装包和相关的配置文件,旨在帮助用户快速搭建和管理Redis服务器。 首先,让我们详细了解一下Redis的基本概念和特性。Redis是一个基于内存的数据结构存储系统,它支持多种数据类型...

    redis集群搭建(linux).zip

    在Linux环境下搭建Redis集群,我们需要了解以下几个关键知识点: 1. **Redis基础**:首先,你需要理解Redis的基本概念,如数据类型(字符串、哈希、列表、集合、有序集合)、持久化机制(RDB和AOF)以及主从复制。 ...

    redis集群搭建以及脚本启动

    Redis 集群搭建与脚本启动是分布式缓存系统中的一个重要话题,它涉及到如何配置多个Redis实例以实现数据的高可用性和可扩展性。在本文中,我们将深入探讨Redis集群的基本概念、搭建过程以及使用脚本自动化启动的方法...

    windowsredis集群搭建

    在Windows环境下搭建Redis集群虽然相比Linux环境复杂一些,但遵循以上步骤,配合相应的调试工具和日志分析,可以顺利完成。记得在生产环境中,要考虑到安全策略、性能优化以及持久化策略等因素,确保集群稳定可靠。

    linux redis安装(redis-2.6.14.tar.gz)

    在Linux系统中安装Redis是一个常见的任务,特别是在搭建分布式缓存或数据库系统时。Redis是一款开源、高性能、基于键值对的数据存储系统,广泛应用于数据缓存、消息队列和数据库等多个场景。本教程将详细讲解如何在...

    redis-cluster redis伪集群文件

    搭建redis集群文件,内含六个redis,文件配置已经全部修改好了,1、先在服务器安装下yum install ruby 和 yum install rubygems 2、然后解压文件上传到linux服务器 3、./redis-cluster-start-all.sh 开启reids 4、...

    基于Redis6的Redis集群搭建方法

    随着业务的发展,单机Redis可能无法满足高可用性和扩展性的需求,这时我们需要搭建一个Redis集群来解决这些问题。本文将详细讲解如何基于Redis 6搭建一个稳定的集群。 **一、Redis集群简介** Redis集群(Redis ...

Global site tag (gtag.js) - Google Analytics