`

[转]基于Twemproxy的Redis集群方案

 
阅读更多

转:http://www.cnblogs.com/haoxinyue/p/redis.html

概述

由于单台redis服务器的内存管理能力有限,使用过大内存redis服务器的性能急剧下降,且服务器发生故障将直接影响大面积业务。为了获取更好的缓存性能及扩展型,我们将需要搭建redis集群来满足需求。因redis 3.0 beta支持的集群功能不适合生产环境的使用,所以我们采用twitter正在使用的twemproxy来搭建redis缓存服务器集群,目前用户包括Pinterest、Tumblr、Twitter、Vine、Kiip、Wuaki.tv、Wanelo、Kontera、Wikimedia、Bright、56.com、Snapchat、Digg、Gawkermedia、3scale.net等。

Twemproxy是memcached和redis协议的代理服务器,并能有效减少大量连接对redis服务器的性能影响,它提供的主要特性如下:

image

 

 

集群架构

image

 

安装Redis

有三台服务器,一台COS1安装twemproxy,另外两台COS2,COS3安装redis。

 

  1. 下载最新安装包:redis-2.8.9.tar.gz , tcl-8.5.7-6.el6.x86_64.rpm ,nutcracker-0.3.0.tar.gz
  2. image
  3. 安装必要组件rpm:
    [root@COS2 redis-2.8.9]# yum install gcc
    [root@COS2 src]# rpm -ivh tcl-8.5.7-6.el6.x86_64.rpm
  4. 安装Redis:
    复制代码
    [root@COS2 src]# tar xvf redis-2.8.9.tar.gz
    [root@COS2 src]# cd redis-2.8.9
    [root@COS2 redis-2.8.9]# make
    …
    Hint: To run 'make test' is a good idea ;)
    make[1]: Leaving directory `/usr/local/src/redis-2.8.9/src'
    
    [root@COS2 redis-2.8.9]# make test
    All tests passed without errors!
    
    Cleanup: may take some time... OK
    make[1]: Leaving directory `/usr/local/src/redis-2.8.9/src'
    [root@COS2 redis-2.8.9]# make install
    [root@COS2 redis-2.8.9]# cd /usr/local/bin/
    [root@COS2 bin]# ll
    total 13908
    -rwxr-xr-x. 1 root root 4170264 Apr 26 11:51 redis-benchmark
    -rwxr-xr-x. 1 root root   22185 Apr 26 11:51 redis-check-aof
    -rwxr-xr-x. 1 root root   45419 Apr 26 11:51 redis-check-dump
    -rwxr-xr-x. 1 root root 4263471 Apr 26 11:51 redis-cli
    -rwxr-xr-x. 1 root root 5726791 Apr 26 11:51 redis-server
    复制代码
  5. 编辑redis配置文件:
    复制代码
    [root@COS2 redis-2.8.9]# cp redis.conf /etc/
    [root@COS2 redis-2.8.9]# vim /etc/red
    redhat-release  redis.conf      
    [root@COS2 redis-2.8.9]# vim /etc/redis.conf
    
    把里面的
    daemonize no  修改成 daemonize yes
    复制代码
  6. 启动redis服务:
    [root@COS2 redis-2.8.9]# redis-server /etc/redis.conf
  7. 测试redis服务:
    [root@COS2 redis-2.8.9]# redis-cli 
    127.0.0.1:6379> set kin kin
    OK
    127.0.0.1:6379> get kin
  8. 同样的步骤安装其他redis服务器。

 

 

安装twemproxy

  1. 安装twemproxy:
    [root@COS1 src]# tar xvf nutcracker-0.3.0.tar.gz
    [root@COS1 nutcracker-0.3.0]# cd nutcracker-0.3.0
    [root@COS1 src]#./configure 
    [root@COS1 nutcracker-0.3.0]# make && make install
  2. 编辑配置文件:
    复制代码
    [root@COS1 conf]# cd /usr/local/src/nutcracker-0.3.0/conf
    [root@COS1 conf]# cp nutcracker.yml /etc/
    [root@COS1 conf]# vim /etc/nutcracker.yml
    alpha:
      listen: 0.0.0.0:22121
      hash: fnv1a_64
      distribution: ketama
      auto_eject_hosts: true
      redis: true
      server_retry_timeout: 2000
      server_failure_limit: 1
      servers: --两台redis服务器的地址和端口
       - 10.23.22.240:6379:1   
       - 10.23.22.241:6379:1
    复制代码
  3. 测试配置文件:
    [root@COS1 nutcracker-0.3.0]# nutcracker -t /etc/nutcracker.yml 
    nutcracker: configuration file 'conf/nutcracker.yml' syntax is ok
  4. 启动twemproxy:
    复制代码
    [root@COS1 nutcracker-0.3.0]# nutcracker  --help
    This is nutcracker-0.3.0
    
    Usage: nutcracker [-?hVdDt] [-v verbosity level] [-o output file]
                      [-c conf file] [-s stats port] [-a stats addr]
                      [-i stats interval] [-p pid file] [-m mbuf size]
    
    Options:
      -h, --help             : this help                           
      -V, --version          : show version and exit                 
      -t, --test-conf        : test configuration for syntax errors and exit 
      -d, --daemonize      : run as a daemon                    
      -D, --describe-stats   : print stats description and exit
      -v, --verbosity=N      : set logging level (default: 5, min: 0, max: 11)
      -o, --output=S         : set logging file (default: stderr)
      -c, --conf-file=S      : set configuration file (default: conf/nutcracker.yml) #配置
      -s, --stats-port=N     : set stats monitoring port (default: 22222)
      -a, --stats-addr=S     : set stats monitoring ip (default: 0.0.0.0)
      -i, --stats-interval=N : set stats aggregation interval in msec (default: 30000 msec)
      -p, --pid-file=S       : set pid file (default: off)
      -m, --mbuf-size=N      : set size of mbuf chunk in bytes (default: 16384 bytes)
    [root@COS1 nutcracker-0.3.0]# nutcracker -d -c /etc/nutcracker.yml
    [root@COS1 nutcracker-0.3.0]# ps -ef|grep nutcracker
    root     15358     1  0 02:40 ?        00:00:00 nutcracker -d -c /etc/nutcracker.yml
    复制代码
  5. 测试twemproxy:
    复制代码
    [root@COS1 ~]# redis-cli -p 22121
    127.0.0.1:22121> get kin
    "kin"
    127.0.0.1:22121> set kin king
    OK
    127.0.0.1:22121> get kin
    "king"
    复制代码

 

 

 

 

性能测试

这里使用redis自带的redis-benchmark进行简单的性能测试,测试结果如下:

  1. Set测试:
    1. 通过twemproxy测试:
      [root@COS1 src]# redis-benchmark -h 10.23.22.240 -p 22121 -c 100 -t set -d 100 -l –q
      
      
      SET: 38167.94 requests per second
    2. 直接对后端redis测试:
      [root@COS2 ~]# redis-benchmark -h 10.23.22.241 -p 6379 -c 100 -t set -d 100 -l –q
      
      
      SET: 53191.49 requests per second
  2. Get测试:
    1. 通过twemproxy测试:
      [root@COS1 src]# redis-benchmark -h 10.23.22.240 -p 22121 -c 100 -t get -d 100 -l -q
      GET: 37453.18 requests per second
    2. 直接对后端redis测试:
      [root@COS2 ~]# redis-benchmark -h 10.23.22.241 -p 6379 -c 100 -t get -d 100 -l -q
      GET: 62111.80 requests per second
  3. 查看键值分布:
    [root@COS2 ~]# redis-cli info|grep db0
    db0:keys=51483,expires=0,avg_ttl=0
    
    [root@COS3 ~]# redis-cli info|grep db0
    db0:keys=48525,expires=0,avg_ttl=0

测试结果:以基本的set get命令通过twemproxy性能有所下降;通过twemproxy分布基本平均。测试数据以业务测试为准。

 

 

---附 twemproxy 支持的redis命令:

https://raw.githubusercontent.com/twitter/twemproxy/master/notes/redis.md

 

## Redis Command Support

### Keys Command

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |        DEL        |    Yes     | DEL key [key …]                                                                                                     |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       DUMP        |    Yes     | DUMP key                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      EXISTS       |    Yes     | EXISTS key                                                                                                          |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      EXPIRE       |    Yes     | EXPIRE key seconds                                                                                                  |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     EXPIREAT      |    Yes     | EXPIREAT key timestamp                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       KEYS        |    No      | KEYS pattern                                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      MIGRATE      |    No      | MIGRATE host port key destination-db timeout                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       MOVE        |    No      | MOVE key db                                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      OBJECT       |    No      | OBJECT subcommand [arguments [arguments …]]                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      PERSIST      |    Yes     | PERSIST key                                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      PEXPIRE      |    Yes     | PEXPIRE key milliseconds                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     PEXPIREAT     |    Yes     | PEXPIREAT key milliseconds-timestamp                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      PTTL         |    Yes     | PTTL key                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     RANDOMKEY     |    No      | RANDOMKEY                                                                                                           |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      RENAME       |    No      | RENAME key newkey                                                                                                   |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     RENAMENX      |    No      | RENAMENX key newkey                                                                                                 |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      RESTORE      |    Yes     | RESTORE key ttl serialized-value                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SORT         |    Yes     | SORT key [BY pattern] [LIMIT offset count] [GET pattern [GET pattern ...]] [ASC|DESC] [ALPHA] [STORE destination]   |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       TTL         |    Yes     | TTL key                                                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      TYPE         |    Yes     | TYPE key                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SCAN         |    No      | SCAN cursor [MATCH pattern] [COUNT count]                                                                           |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

### Strings Command

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       APPEND      |    Yes     | APPEND key value                                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      BITCOUNT     |    Yes     | BITCOUNT key [start] [end]                                                                                          |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      BITPOS       |    Yes     | BITPOS key bit [start] [end]                                                                                          |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       BITOP       |    No      | BITOP operation destkey key [key ...]                                                                               |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       DECR        |    Yes     | DECR key                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      DECRBY       |    Yes     | DECRBY key decrement                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       GET         |    Yes     | GET key                                                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      GETBIT       |    Yes     | GETBIT key offset                                                                                                   |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     GETRANGE      |    Yes     | GETRANGE key start end                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      GETSET       |    Yes     | GETSET key value                                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      INCR         |    Yes     | INCR key                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      INCRBY       |    Yes     | INCRBY key increment                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     INCRBYFLOAT   |    Yes     | INCRBYFLOAT key increment                                                                                           |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      MGET         |    Yes     | MGET key [key ...]                                                                                                  |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      MSET         |    Yes*    | MSET key value [key value ...]                                                                                      | 
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      MSETNX       |    No      | MSETNX key value [key value ...]                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      PSETEX       |    Yes     | PSETEX key milliseconds value                                                                                       |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SET          |    Yes     | SET key value [EX seconds] [PX milliseconds] [NX|XX]                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SETBIT       |    Yes     | SETBIT key offset value                                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SETEX        |    Yes     | SETEX key seconds value                                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SETNX        |    Yes     | SETNX key value                                                                                                     |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SETRANGE     |    Yes     | SETRANGE key offset value                                                                                           |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      STRLEN       |    Yes     | STRLEN key                                                                                                          |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

* MSET support is not Atomic 

### Hashes

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       HDEL        |    Yes     | HDEL key field [field ...]                                                                                          |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HEXISTS      |    Yes     | HEXISTS key field                                                                                                   |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       HGET        |    Yes     | HGET key field                                                                                                      |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HGETALL      |    Yes     | HGETALL key                                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HINCRBY      |    Yes     | HINCRBY key field increment                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    HINCRBYFLOAT   |    Yes     | HINCRBYFLOAT key field increment                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HKEYS        |    Yes     | HKEYS key                                                                                                           |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HLEN         |    Yes     | HLEN key                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HMGET        |    Yes     | HMGET key field [field ...]                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HMSET        |    Yes     | HMSET key field value [field value ...]                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HSET         |    Yes     | HSET key field value                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HSETNX       |    Yes     | HSETNX key field value                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HVALS        |    Yes     | HVALS key                                                                                                           |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      HSCAN        |    Yes     | HSCAN key cursor [MATCH pattern] [COUNT count]                                                                      |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

### Lists

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       BLPOP       |    No      | BLPOP key [key ...] timeout                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       BRPOP       |    No      | BRPOP key [key ...] timeout                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     BRPOPLPUSH    |    No      | BRPOPLPUSH source destination timeout                                                                               |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      LINDEX       |    Yes     | LINDEX key index                                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      LINSERT      |    Yes     | LINSERT key BEFORE|AFTER pivot value                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      LLEN         |    Yes     | LLEN key                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      LPOP         |    Yes     | LPOP key                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      LPUSH        |    Yes     | LPUSH key value [value ...]                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      LPUSHX       |    Yes     | LPUSHX key value                                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      LRANGE       |    Yes     | LRANGE key start stop                                                                                               |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      LREM         |    Yes     | LREM key count value                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      LSET         |    Yes     | LSET key index value                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      LTRIM        |    Yes     | LTRIM key start stop                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      RPOP         |    Yes     | RPOP key                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     RPOPLPUSH     |    Yes*    | RPOPLPUSH source destination                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      RPUSH        |    Yes     | RPUSH key value [value ...]                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      RPUSHX       |    Yes     | RPUSHX key value                                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

* RPOPLPUSH support requires that source and destination keys hash to the same server. You can ensure this by using the same [hashtag](recommendation.md#hash-tags) for source and destination key. Twemproxy does no checking on its end to verify that source and destination key hash to the same server, and the RPOPLPUSH command is forwarded to the server that the source key hashes to

### Sets

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SADD         |    Yes     | SADD key member [member ...]                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SCARD        |    Yes     | SCARD key                                                                                                           |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SDIFF        |    Yes*    | SDIFF key [key ...]                                                                                                 |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     SDIFFSTORE    |    Yes*    | SDIFFSTORE destination key [key ...]                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SINTER       |    Yes*    | SINTER key [key ...]                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    SINTERSTORE    |    Yes*    | SINTERSTORE destination key [key ...]                                                                               |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     SISMEMBER     |    Yes     | SISMEMBER key member                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     SMEMBERS      |    Yes     | SMEMBERS key                                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SMOVE        |    Yes*    | SMOVE source destination member                                                                                     |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SPOP         |    Yes     | SPOP key                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    SRANDMEMBER    |    Yes     | SRANDMEMBER key [count]                                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SREM         |    Yes     | SREM key member [member ...]                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     SUNION        |    Yes*    | SUNION key [key ...]                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |   SUNIONSTORE     |    Yes*    | SUNIONSTORE destination key [key ...]                                                                               |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SSCAN        |    Yes     | SSCAN key cursor [MATCH pattern] [COUNT count]                                                                      |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

* SIDFF, SDIFFSTORE, SINTER, SINTERSTORE, SMOVE, SUNION and SUNIONSTORE support requires that the supplied keys hash to the same server. You can ensure this by using the same [hashtag](recommendation.md#hash-tags) for all keys in the command. Twemproxy does no checking on its end to verify that all the keys hash to the same server, and the given command is forwarded to the server that the first key hashes to.


### Sorted Sets

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      ZADD         |    Yes     | ZADD key score member [score] [member]                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      ZCARD        |    Yes     | ZCARD key                                                                                                           |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      ZCOUNT       |    Yes     | ZCOUNT key min max                                                                                                  |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      ZINCRBY      |    Yes     | ZINCRBY key increment member                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     ZINTERSTORE   |    Yes*    | ZINTERSTORE destination numkeys key [key ...] [WEIGHTS weight [weight ...]] [AGGREGATE SUM|MIN|MAX]                 |
    +------------------------------------------------------------------------------------------------------------------------------------------------------+
    |      ZLEXCOUNT    |    Yes     | ZLEXCOUNT key min max                                                                                               |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      ZRANGE       |    Yes     | ZRANGE key start stop [WITHSCORES]                                                                                  |
    +------------------------------------------------------------------------------------------------------------------------------------------------------+
    |    ZRANGEBYLEX    |    Yes     | ZRANGEBYLEX key min max [LIMIT offset count]                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    ZRANGEBYSCORE  |    Yes     | ZRANGEBYSCORE key min max [WITHSCORES] [LIMIT offset count]                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      ZRANK        |    Yes     | ZRANK key member                                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       ZREM        |    Yes     | ZREM key member [member ...]                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |   ZREMRANGEBYLEX  |    Yes     | ZREMRANGEBYLEX key min max                                                                                          |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |   ZREMRANGEBYRANK |    Yes     | ZREMRANGEBYRANK key start stop                                                                                      |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |  ZREMRANGEBYSCORE |    Yes     | ZREMRANGEBYSCORE key min max                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    ZREVRANGE      |    Yes     | ZREVRANGE key start stop [WITHSCORES]                                                                               |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |  ZREVRANGEBYSCORE |    Yes     | ZREVRANGEBYSCORE key max min [WITHSCORES] [LIMIT offset count]                                                      |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     ZREVRANK      |    Yes     | ZREVRANK key member                                                                                                 |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     ZSCORE        |    Yes     | ZSCORE key member                                                                                                   |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    ZUNIONSTORE    |    Yes*    | ZUNIONSTORE destination numkeys key [key ...] [WEIGHTS weight [weight ...]] [AGGREGATE SUM|MIN|MAX]                 |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      ZSCAN        |    Yes     | ZSCAN key cursor [MATCH pattern] [COUNT count]                                                                      |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

* ZINTERSTORE and ZUNIONSTORE support requires that the supplied keys hash to the same server. You can ensure this by using the same [hashtag](recommendation.md#hash-tags) for all keys in the command. Twemproxy does no checking on its end to verify that all the keys hash to the same server, and the given command is forwarded to the server that the first key hashes to.

### HyperLogLog

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       PFADD       |    Yes     | PFADD key element [element ...]                                                                                     |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      PFCOUNT      |    Yes     | PFCOUNT key [key ...]                                                                                               |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      PFMERGE      |    Yes*    | PFMERGE destkey sourcekey [sourcekey ...]                                                                           |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

* PFMERGE support requires that the supplied keys hash to the same server. You can ensure this by using the same [hashtag](recommendation.md#hash-tags) for all keys in the command. Twemproxy does no checking on its end to verify that all the keys hash to the same server, and the given command is forwarded to the server that the first key hashes to.


### Pub/Sub

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     PSUBSCRIBE    |    No      | PSUBSCRIBE pattern [pattern ...]                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     PUBLISH       |    No      | PUBLISH channel message                                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    PUNSUBSCRIBE   |    No      | PUNSUBSCRIBE [pattern [pattern ...]]                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     SUBSCRIBE     |    No      | SUBSCRIBE channel [channel ...]                                                                                     |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     UNSUBSCRIBE   |    No      | UNSUBSCRIBE [channel [channel ...]]                                                                                 |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

### Transactions

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      DISCARD      |    No      | DISCARD                                                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       EXEC        |    No      | EXEC                                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       MULTI       |    No      | MULTI                                                                                                               |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      UNWATCH      |    No      | UNWATCH                                                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       WATCH       |    No      | WATCH key [key ...]                                                                                                 |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

### Scripting

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       EVAL        |    Yes*    | EVAL script numkeys key [key ...] arg [arg ...]                                                                     |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     EVALSHA       |    Yes*    | EVALSHA sha1 numkeys key [key ...] arg [arg ...]                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    SCRIPT EXISTS  |    No      | SCRIPT EXISTS script [script ...]                                                                                   |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    SCRIPT FLUSH   |    No      | SCRIPT FLUSH                                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    SCRIPT KILL    |    No      | SCRIPT KILL                                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    SCRIPT LOAD    |    No      | SCRIPT LOAD script                                                                                                  |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

 * EVAL and EVALSHA support is limited to scripts that take at least 1 key. If multiple keys are used, all keys must hash to the same server. You can ensure this by using the same [hashtag](recommendation.md#hash-tags) for all keys. If you use more than 1 key, the proxy does no checking to verify that all keys hash to the same server, and the entire command is forwarded to the server that the first key hashes to

### Connection

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       AUTH        |    No      | AUTH password                                                                                                       |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       ECHO        |    No      | ECHO message                                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       PING        |    No      | PING                                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |       QUIT        |    No      | QUIT                                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SELECT       |    No      | SELECT index                                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

### Server

    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      Command      | Supported? | Format                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    BGREWRITEAOF   |    No      | BGREWRITEAOF                                                                                                        |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      BGSAVE       |    No      | BGSAVE                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    CLIENT KILL    |    No      | CLIENT KILL ip:port                                                                                                 |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    CLIENT LIST    |    No      | CLIENT LIST                                                                                                         |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    CONFIG GET     |    No      | CONFIG GET parameter                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    CONFIG SET     |    No      | CONFIG SET parameter value                                                                                          |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |  CONFIG RESETSTAT |    No      | CONFIG RESETSTAT                                                                                                    |
    +-------------------+-------------+--------------------------------------------------------------------------------------------------------------------+
    |     DBSIZE        |    No      | DBSIZE                                                                                                              |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    DEBUG OBJECT   |    No      | DEBUG OBJECT key                                                                                                    |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |    DEBUG SEGFAULT |    No      | DEBUG SEGFAULT                                                                                                      |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     FLUSHALL      |    No      | FLUSHALL                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     FLUSHDB       |    No      | FLUSHDB                                                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      INFO         |    No      | INFO                                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     LASTSAVE      |    No      | LASTSAVE                                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     MONITOR       |    No      | MONITOR                                                                                                             |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SAVE         |    No      | SAVE                                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     SHUTDOWN      |    No      | SHUTDOWN [NOSAVE] [SAVE]                                                                                            |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     SLAVEOF       |    No      | SLAVEOF host port                                                                                                   |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |     SLOWLOG       |    No      | SLOWLOG subcommand [argument]                                                                                       |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      SYNC         |    No      | SYNC                                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
    |      TIME         |    No      | TIME                                                                                                                |
    +-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

## Note

- redis commands are not case sensitive
- only vectored commands 'MGET key [key ...]', 'MSET key value [key value ...]', 'DEL key [key ...]' needs to be fragmented

## Performance

### Setup

+ redis-server running on machine A.
+ nutcracker running on machine A as a local proxy to redis-server.
+ redis-benchmark running on machine B.
+ machine A != machine B.
+ nutcracker built with --enable-debug=no
+ nutcracker running with mbuf-size of 512 (-m 512)
+ redis-server built from redis 2.6 branch

### redis-benchmark against redis-server

    $ redis-benchmark -h <machine-A> -q -t set,get,incr,lpush,lpop,sadd,spop,lpush,lrange -c 100 -p 6379
    SET: 89285.71 requests per second
    GET: 92592.59 requests per second
    INCR: 89285.71 requests per second
    LPUSH: 90090.09 requests per second
    LPOP: 90090.09 requests per second
    SADD: 90090.09 requests per second
    SPOP: 93457.95 requests per second
    LPUSH (needed to benchmark LRANGE): 89285.71 requests per second
    LRANGE_100 (first 100 elements): 36496.35 requests per second
    LRANGE_300 (first 300 elements): 15748.03 requests per second
    LRANGE_500 (first 450 elements): 11135.86 requests per second
    LRANGE_600 (first 600 elements): 8650.52 requests per second

### redis-benchmark against nutcracker proxing redis-server

    $ redis-benchmark -h <machine-A> -q -t set,get,incr,lpush,lpop,sadd,spop,lpush,lrange -c 100 -p 22121
    SET: 85470.09 requests per second
    GET: 86956.52 requests per second
    INCR: 85470.09 requests per second
    LPUSH: 84745.77 requests per second
    LPOP: 86206.90 requests per second
    SADD: 84745.77 requests per second
    SPOP: 86956.52 requests per second
    LPUSH (needed to benchmark LRANGE): 84745.77 requests per second
    LRANGE_100 (first 100 elements): 29761.90 requests per second
    LRANGE_300 (first 300 elements): 12376.24 requests per second
    LRANGE_500 (first 450 elements): 8605.85 requests per second
    LRANGE_600 (first 600 elements): 6587.62 requests per second

## redis-auth feature

+ you can enable redis-auth for a pool with 'redis_auth':

        alpha:
          listen: 127.0.0.1:22121
          hash: fnv1a_64
          distribution: ketama
          redis: true
          redis_auth: testpass

+ notice:
    + *MUST* set all redis with a same passwd, and all twemproxy with the same passwd
    + Length of password should less than 256 

 

 

 

 

 

 

 

 

 

分享到:
评论

相关推荐

    基于代理的 Redis 集群解决方案,支持管道和动态扩展.zip

    Codis 是一个基于代理的高性能 Redis 集群解决方案,使用 Go 编写。它已投入生产,并在wandoujia.com和许多公司中广泛使用。您可以查看Codis Releases了解最新和最稳定的版本。捐款如果您想帮助我们维护这个项目,请...

    redis集群搭建及各方案比较

    4. Redisa:基于 Redis Sentinel 的集群方案,侧重高可用性,但不支持数据分片。 选择哪种方案取决于具体业务需求,如数据一致性、高可用性、运维便捷性等因素。 总之,Redis 集群搭建是一个综合考虑数据分布、...

    Redis持久化以及集群部署

    除了官方提供的集群方案外,还有一些第三方工具可以用于 Redis 的集群部署,例如 Twemproxy 和 Codis。 - **Twemproxy**: - 优点:配置简单,支持多种数据库; - 缺点:不支持动态扩容,运维相对麻烦; - **...

    Redis 集群解决方案 Codis.zip

    codis-config 本身还自带了一个 http server, 会启动一个 dashboard, 用户可以直接在浏览器上观察 Codis 集群的运行状态.codis-server 是 Codis 项目维护的一个 Redis 分支, 基于 2.8.13 开发, 加入了 slot 的支持...

    最全50道Redis高频面试题.pdf

    Redis集群方案主要包括twemproxy、codis和Redis自带的集群解决方案。twemproxy是一种代理方式,通过一致性哈希算法分发请求到多个Redis实例,并支持自动节点故障转移。codis是另一个流行的集群方案,它解决了...

    Redis面试题(含答案)_20190220222533.pdf

    Redis集群方案主要有以下几种: 1. **Twemproxy**:类似于代理服务器的角色,使用简单,但存在单点压力问题。 2. **Codis**:与Twemproxy相似,但在节点变化时能自动调整数据分布。 3. **Redis Cluster 3.0**:内置...

    完全兼容redis协议的分布式数据库.zip

    完全兼容redis协议的分布式数据库#Reborn——Redis 的另一个快速分布式解决方案Reborn 是一个用 Go/C 编写的基于代理的高性能 Redis 集群解决方案,是 Redis 的替代品。Reborn 支持多个无状态代理和多个 redis 实例...

    Redis面试专题1

    在实现 Redis 集群方案时,有以下几种常见方法: 1. twemproxy:作为一个代理,它接受客户端的请求,使用一致性哈希将请求分发到多个 Redis 实例。优点是配置简单,但存在单点压力和数据迁移不便的问题。 2. Codis:...

    50道Redis面试题及答案冲击百万年薪.docx

    对于 Redis 集群方案,有以下几种常见选择: 1. twemproxy:作为代理服务器,使用一致性哈希分配数据,但不支持数据迁移,且单点压力较大。 2. Codis:提供了更好的数据迁移功能,允许动态调整集群节点。 3. Redis ...

    Redis的改造与实践

    随着Nice服务端架构的快速发展,原有的Redis集群面临着前所未有的挑战。项目初期采取的是快速而粗犷的发展方式,这导致了数百台服务器支撑着上千个Redis实例。随着业务量的增长,对集群管理效率的要求也日益提高。...

    redis集群架构

    总之,Redis集群架构提供了高可用性和可扩展性,但不同的架构有不同的优缺点,需要根据实际需求和场景选择最合适的方案。随着技术的发展,Redis Cluster已经成为更推荐的集群解决方案,因为它能够提供更强大的功能和...

    Java面试题redis部分.docx

    面试中,Redis的相关问题通常涉及到其架构、数据一致性、集群方案等关键点。 1. Redis 主从架构: 主从复制是Redis提供的一种高可用性方案,通过将数据复制到多个从节点,实现数据冗余和读负载均衡。主节点负责写...

    5_redis.docx

    - **TwemProxy**:这是一个代理层组件,主要用于管理Redis集群,它可以减少客户端与Redis服务器之间的直接连接数量,提高系统的整体性能。 - **3.0 Cluster**:自Redis 3.0版本开始,官方提供了内置的集群支持。集群...

    Codis分布式Redis解决方案-其他

    Codis是一个分布式Redis解决方案,对于上层的应用来说,连接到CodisProxy和连接原生的RedisServer没有明显的区别(不支持的命令列表),上层应用可以像使用单机的Redis一样使用,Codis底层会处理请求的转发,不停机的...

    Redis面试题汇总经典.docx

    Redis集群方案 - **Twemproxy**: 作为代理服务器,通过一致性hash算法分发请求到不同的Redis实例。易于集成,但存在单点压力和节点变动时数据迁移的问题。 - **Codis**: 提供了更完善的解决方案,支持节点数量变化...

    Redis 50 道面试题及答案.docx

    Redis 有多种集群方案,如: * twemproxy:使用代理方式来实现集群。 * Codis:使用一致性哈希来实现集群。 * Redis cluster 3.0:使用分布式算法来实现集群。 * 业务代码层实现:在业务代码层面实现集群。 标题:...

    redis面试专题.pdf

    1. **codis:**一种开源的Redis集群方案,支持动态调整节点数量。 2. **Twemproxy:**类似于codis,支持一致性哈希,但在节点变化时需手动调整数据分布。 3. **Redis Cluster:**Redis 3.0自带的集群模式,使用hash...

Global site tag (gtag.js) - Google Analytics