【twemproxy 介绍】

twemproxy,也叫nutcraker。是一个twtter开源的一个redis和memcache代理服务器。 redis作为一个高效的缓存服务器，非常具有应用价值。但是当使用比较多的时候，就希望可以通过某种方式 统一进行管理。避免每个应用每个客户端管理连接的松散性。同时在一定程度上变得可以控制。

 

Twemproxy是一种代理分片机制，由Twitter开源。Twemproxy作为代理，可接受来自多个程序的访问，按照路由规则，转发给后台的各个Redis服务器，再原路返回。该方案很好的解决了单个Redis实例承载能力的问题。当然，Twemproxy本身也是单点，需要用Keepalived做高可用方案。通过Twemproxy可以使用多台服务器来水平扩张redis服务，可以有效的避免单点故障问题。虽然使用Twemproxy需要更多的硬件资源和在redis性能有一定的损失（twitter测试约20%），但是能够提高整个系统的HA也是相当划算的。不熟悉twemproxy的同学，如果玩过nginx反向代理或者mysql proxy，那么你肯定也懂twemproxy了。其实twemproxy不光实现了redis协议，还实现了memcached协议，什么意思？换句话说，twemproxy不光可以代理redis，还可以代理memcached，官方说明：

twemproxy (pronounced "two-em-proxy"), aka nutcracker is a fast and lightweight proxy for memcachedand redis protocol. It was built primarily to reduce the number of connections to the caching servers on the backend. This, together with protocol pipeling and sharding enables you to horizontally scale your distributed caching architecture.

 

Features

Fast.

Lightweight.

Maintains persistent server connections.

Keeps connection count on the backend caching servers low.

Enables pipelining of requests and responses.

Supports proxying to multiple servers.

Supports multiple server pools simultaneously.

Shard data automatically across multiple servers.

Implements the complete memcached ascii and redis protocol.

Easy configuration of server pools through a YAML file.

Supports multiple hashing modes including consistent hashing and distribution.

Can be configured to disable nodes on failures.

Observability via stats exposed on the stats monitoring port.

Works with Linux, *BSD, OS X and SmartOS (Solaris)

 

 

twemproxy

• 支持失败节点自动删除

  • 可以设置重新连接该节点的时间
  • 可以设置连接多少次之后删除该节点
  • 该方式适合作为cache存储

• 支持设置HashTag

  • 通过HashTag可以自己设定将两个KEYhash到同一个实例上去。

• 减少与redis的直接连接数

  • 保持与redis的长连接
  • 可设置代理与后台每个redis连接的数目

• 自动分片到后端多个redis实例上

  • 多种hash算法（部分还没有研究明白)
  • 可以设置后端实例的权重

• 避免单点问题

  • 可以平行部署多个代理层.client自动选择可用的一个

• 支持redis pipelining request

• 支持状态监控

  • 可设置状态监控ip和端口，访问ip和端口可以得到一个json格式的状态信息串
  • 可设置监控信息刷新间隔时间

• 高吞吐量

• 连接复用，内存复用。
• 将多个连接请求，组成reids pipelining统一向redis请求。

Zero Copy

In twemproxy, all the memory for incoming requests and outgoing responses is allocated in mbuf. Mbuf enables zero-copy because the same buffer on which a request was received from the client is used for forwarding it to the server. Similarly the same mbuf on which a response was received from the server is used for forwarding it to the client.

Furthermore, memory for mbufs is managed using a reuse pool. This means that once mbuf is allocated, it is not deallocated, but just put back into the reuse pool. By default each mbuf chunk is set to 16K bytes in size. There is a trade-off between the mbuf size and number of concurrent connections twemproxy can support. A large mbuf size reduces the number of read syscalls made by twemproxy when reading requests or responses. However, with a large mbuf size, every active connection would use up 16K bytes of buffer which might be an issue when twemproxy is handling large number of concurrent connections from clients. When twemproxy is meant to handle a large number of concurrent client connections, you should set chunk size to a small value like 512 bytes using the -m or --mbuf-size=N argument.

Configuration

Twemproxy can be configured through a YAML file specified by the -c or --conf-file command-line argument on process start. The configuration file is used to specify the server pools and the servers within each pool that twemproxy manages. The configuration files parses and understands the following keys:

• listen: The listening address and port (name:port or ip:port) or an absolute path to sock file (e.g. /var/run/nutcracker.sock) for this server pool.
• client_connections: The maximum number of connections allowed from redis clients. Unlimited by default, though OS-imposed limitations will still apply.
• hash: The name of the hash function. Possible values are:
• one_at_a_time
• md5
• crc16
• crc32 (crc32 implementation compatible with libmemcached)
• crc32a (correct crc32 implementation as per the spec)
• fnv1_64
• fnv1a_64
• fnv1_32
• fnv1a_32
• hsieh
• murmur
• jenkins
• hash_tag: A two character string that specifies the part of the key used for hashing. Eg "{}" or "$$". Hash tag enable mapping different keys to the same server as long as the part of the key within the tag is the same.
• distribution: The key distribution mode. Possible values are:
• ketama
• modula
• random
• timeout: The timeout value in msec that we wait for to establish a connection to the server or receive a response from a server. By default, we wait indefinitely.
• backlog: The TCP backlog argument. Defaults to 512.
• preconnect: A boolean value that controls if twemproxy should preconnect to all the servers in this pool on process start. Defaults to false.
• redis: A boolean value that controls if a server pool speaks redis or memcached protocol. Defaults to false.
• redis_auth: Authenticate to the Redis server on connect.
• redis_db: The DB number to use on the pool servers. Defaults to 0. Note: Twemproxy will always present itself to clients as DB 0.
• server_connections: The maximum number of connections that can be opened to each server. By default, we open at most 1 server connection.
• auto_eject_hosts: A boolean value that controls if server should be ejected temporarily when it fails consecutively server_failure_limit times. See liveness recommendations for information. Defaults to false.
• server_retry_timeout: The timeout value in msec to wait for before retrying on a temporarily ejected server, when auto_eject_host is set to true. Defaults to 30000 msec.
• server_failure_limit: The number of consecutive failures on a server that would lead to it being temporarily ejected when auto_eject_host is set to true. Defaults to 2.
• servers: A list of server address, port and weight (name:port:weight or ip:port:weight) for this server pool.