Redis Cluster 2019(3)Redis Cluster on CentOS

Redis Cluster 2019(3)Redis Cluster on CentOS

Current release is still 5.0.5
> sudo yum install wget
> sudo yum groupinstall "Development tools"

>  wget http://download.redis.io/releases/redis-5.0.5.tar.gz

> tar zxvf redis-5.0.5.tar.gz
> cd redis-5.0.5
> make distclean
> make

During make, we may have these issues
jemalloc/jemalloc.h: No such file or directory

If so, we may need to run the make distclean again and then make.

Prepare the installation directory
> mkdir ~/tool/redis-5.0.5

> mkdir ~/tool/redis-5.0.5/bin
> cp src/redis-server ~/tool/redis-5.0.5/bin/
> cp src/redis-cli ~/tool/redis-5.0.5/bin/
> cp src/redis-benchmark ~/tool/redis-5.0.5/bin/

> mkdir ~/tool/redis-5.0.5/conf
> cp redis.conf ~/tool/redis-5.0.5/conf/

Soft link
> sudo ln -s /home/carl/tool/redis-5.0.5 /opt/redis-5.0.5
> sudo ln -s /opt/redis-5.0.5 /opt/redis

Go to the Redis directory
> cd /opt/redis

Prepare the cluster conf
> mkdir cluster-conf
> mkdir cluster-conf/7001
> mkdir cluster-conf/7002

Prepare the default configuration to the cluster
> cp conf/redis.conf cluster-conf/7001/
> cp conf/redis.conf cluster-conf/7002/

Edit the configuration
> vi cluster-conf/7001/redis.conf
port 7001
appendonly yes
cluster-enabled yes
cluster-config-file nodes-7001.conf
cluster-node-timeout 15000
bind 0.0.0.0

> vi cluster-conf/7002/redis.conf
port 7002
appendonly yes
cluster-enabled yes
cluster-config-file nodes-7002.conf
cluster-node-timeout 15000
bind 0.0.0.0

Do the similar things to centos-dev1, centos-dev2, centos-dev3, or we can just copy them to other machines
> scp centos-dev1:/opt/redis/cluster-conf/7001/redis.conf ./cluster-conf/7001/redis.conf

> scp centos-dev1:/opt/redis/cluster-conf/7002/redis.conf ./cluster-conf/7002/redis.conf

Start the redis server on all 3 machines
> bin/redis-server ./cluster-conf/7001/redis.conf
> bin/redis-server ./cluster-conf/7002/redis.conf

After start 6 nodes across 3 machines, we need to init the cluster, my network information is as follow:
192.168.56.102  centos-dev1
192.168.56.111  centos-dev2
192.168.56.112  centos-dev3

On my virtual box machines, I may need to stop the firewall or disable the firewall
> sudo systemctl stop firewalld

> sudo systemctl disable firewalld

> bin/redis-cli --cluster create 192.168.56.102:7001 192.168.56.102:7002 192.168.56.111:7001 192.168.56.111:7002 192.168.56.112:7001 192.168.56.112:7002 --cluster-replicas 1
>>> Performing Cluster Check (using node 192.168.56.102:7001)
M: fc80c3e30badbc2a0e2ec025caaf5709f52090dc 192.168.56.102:7001
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: 6f4aae743cf544c03b00cf91b50c97f61fc72c91 192.168.56.112:7001
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: a32cf75ace9e645d1040699097f3f68965e1f3c9 192.168.56.112:7002
   slots: (0 slots) slave
   replicates de80b1038ab53c35b16ee010f7576ef50788fc81
S: 8f157bb59e184fe04e1a0654cefb7ca72c904234 192.168.56.111:7002
   slots: (0 slots) slave
   replicates fc80c3e30badbc2a0e2ec025caaf5709f52090dc
S: a1f8b4433acf6b8b9a09852aafa028cd4534590e 192.168.56.102:7002
   slots: (0 slots) slave
   replicates 6f4aae743cf544c03b00cf91b50c97f61fc72c91
M: de80b1038ab53c35b16ee010f7576ef50788fc81 192.168.56.111:7001
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

Then the cluster is ready.
Add the bin to the PATH to make my life easier
> vi ~/.bash_profile
PATH=$PATH:/opt/redis/bin

Apply the change
> . ~/.bash_profile

Connect to verify
> redis-cli -c -h centos-dev1 -p 7001
centos-dev1:7001> set foo bar
-> Redirected to slot [12182] located at 192.168.56.112:7001
OK
192.168.56.112:7001> get foo
"bar"
192.168.56.112:7001>

Show all the cluster nodes
192.168.56.112:7001> cluster nodes
6f4aae743cf544c03b00cf91b50c97f61fc72c91 192.168.56.112:7001@17001 myself,master - 0 1565981164000 5 connected 10923-16383
8f157bb59e184fe04e1a0654cefb7ca72c904234 192.168.56.111:7002@17002 slave fc80c3e30badbc2a0e2ec025caaf5709f52090dc 0 1565981165000 4 connected
fc80c3e30badbc2a0e2ec025caaf5709f52090dc 192.168.56.102:7001@17001 master - 0 1565981167640 1 connected 0-5460
a1f8b4433acf6b8b9a09852aafa028cd4534590e 192.168.56.102:7002@17002 slave 6f4aae743cf544c03b00cf91b50c97f61fc72c91 0 1565981166000 5 connected
de80b1038ab53c35b16ee010f7576ef50788fc81 192.168.56.111:7001@17001 master - 0 1565981166000 3 connected 5461-10922
a32cf75ace9e645d1040699097f3f68965e1f3c9 192.168.56.112:7002@17002 slave de80b1038ab53c35b16ee010f7576ef50788fc81 0 1565981166627 6 connected


Some warnings need to care:
Warning #1
5976:M 16 Aug 2019 01:01:24.396 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.

Solution:
https://www.jianshu.com/p/7319c6d6f365
> sudo vi /etc/sysctl.conf
net.core.somaxconn=2048

Verify the change
> sudo sysctl -p
net.core.somaxconn = 2048

Warning #2
5976:M 16 Aug 2019 01:01:24.396 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.

Solution:
https://stackoverflow.com/questions/41203492/solving-redis-warnings-on-overcommit-memory-and-transparent-huge-pages-for-ubunt
> sudo vi /etc/sysctl.conf
vm.overcommit_memory=1
Verify the change
> sudo sysctl -p
net.core.somaxconn = 2048
vm.overcommit_memory = 1

Warning #3
5976:M 16 Aug 2019 01:01:24.396 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.

Solution:
https://github.com/docker-library/redis/issues/55
Need to su to root user to do this
> echo never > /sys/kernel/mm/transparent_hugepage/enabled
> echo never > /sys/kernel/mm/transparent_hugepage/defrag
> cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]
> cat /sys/kernel/mm/transparent_hugepage/defrag
always madvise [never]


References:
https://www.psychz.net/client/question/en/turn-off-firewall-centos-7.html
https://github.com/junegunn/redis-stat
https://www.jianshu.com/p/7319c6d6f365