`
liuxinglanyue
  • 浏览: 561393 次
  • 性别: Icon_minigender_1
  • 来自: 杭州
社区版块
存档分类
最新评论

Redis, from the Ground Up(2)

阅读更多

Key Disadvantages

Redis requires that the whole dataset be loaded into main memory at all times. (Redis Virtual Memory, which we’ll discuss later, relaxes this requirement, but still needs all keys to always be in memory.). Guaranteed in-memory access to most of the dataset is Redis' main performance driver — and is also responsible for creating its main limitations.

RAM

RAM is the gold of cloud computing. (Cloud servers are primarily priced based on the amount of available RAM. By comparison, disk and CPU are cheap.)

The amount of RAM that Redis needs is proportional to the size of the dataset. Large datasets in Redis are going to be fast, but expensive.

Persistence

Redis persistence is highly configurable but the implementation makes extremely heavy use of I/O resources. Furthermore, most save operations require additional memory to complete successfully, and, in some cases, asynchronous saves can block the server for lengthy periods of time. (These points are discussed in more detail, below; see thePersistence section.)

Memory Bloat

Redis' internal design typically trades off memory for speed. For some workloads, there can be an order of magnitude difference between the raw number of bytes handed off to Redis to store, and the amount of memory that Redis uses.

Diving In

String Keys

Regardless of the data type, the data is always identified by a key, and the key is always a string.

For example, using the string data type:

    redis> SET foo bar
    OK
    redis> GET foo
    "bar"
    redis> GET dne
    (nil)

Expiry

Keys can be marked for expiry. For example:

    redis> EXPIRE foo 2
    (integer) 1

After waiting for 2 seconds:

    redis> GET foo
    (nil)

Sidenote: memcached?

So far, this looks quite similar to memcached (GET/SET API, in-memory storage, etc.). However, there are a few important things to note:

  1. Redis supports replication out of the box. Any sort of topology is possible, so you can create replication trees.
  2. Redis supports persistence, so you don’t lose everything that’s in memory when the server restarts.
  3. Redis supports a rich set of data types (far beyond memcached’s simple key-value-pairs).

Each of these points will be addressed in more detail, below.

Replication

Redis' replication capabilities are powerful yet straightforward.

A master can have any number of slaves, and each slave can have any number of their own slaves, and so on and so forth. Any topology is possible.

To point a slave to a specific master, issue the SLAVEOF command on the slave. Slaves will block until the initial synchronization with the master is complete.

This initial synchronization process consists of the master asynchronously snapshotting the current state of the database, then transferring the snapshot to the slave, and then subsequently streaming all commands received after initiating the snapshot.

Persistence

Redis has configurable persistence settings, enabling durability to be tweaked depending on the problem domain.

Options

If durability is not important:

Redis can be configured in “snapshotting mode”. In this mode, Redis saves a binary dump of the contents of the database every x seconds or every y operations. If one of these criteria are met, Redis forks the process. The child process writes the dump file to disk while the master continues to service requests.

This procedure can be memory-efficient due to the way that Copy-On-Write works when forking. (Here, a snapshot of the database is saved as it existed exactly at the time of forking; extra memory is required only to store the keys that change during the snapshot procedure. If every key changes in value over the course of the snapshot, then roughly 2x the amount of memory used by Redis before the save is required to complete the save operation. This is the upper bound on the memory usage required for saving.)

Of course, in this mode, any data that is not written in the snapshot is immediately lost if the server is killed.

If durability is important:

Redis can be configured to use an Append-Only File (AOF). Here, every command is written to a file. To recover from a crash or other server restart, the append-only file is replayed. There are three modes:

  • fsync() on every new command
  • fsync() every second
  • Let the OS decide when to fysnc()

Using the BGREWRITEAOF command, Redis will update the snapshot and re-write the Append-Only File to shorten it. Like snapshotting, this is done asynchronously, in the background.

More advanced configurations:

Persistence can be turned off completely. This is useful in a number of scenarios.

For example, if performance is very critical and your application demands extremely tight control over RAM usage, the following configuration is possible:

  • One master, persistence off, and
  • One slave, persistence off, and
  • Periodic synchronous saves, issues against the slave only

The advantage of this set-up is that it requires no extra memory to complete a save, regardless of the number and frequency of writes. In this way, you are trading off durability for extremely tight control over memory usage.

No extra memory is required to complete the save because the SAVE command performs a synchronous save operation, thereby blocking the server that the command is issued against until the saving process completes. (Asynchronous saves, as discussed above, require extra memory proportional to the number of writes performed during the save.)

Other variations on this theme are possible, for example AOF can be enabled on the slave only while persistence remains off on the master.

Binary Dumps and In-Memory Representation

The binary dumps (i.e. those produced by the snapshot operations) are stored in a very efficient manner on disk.

Once a binary dump is loaded, Redis will use several factors more memory than the on-disk representation requires. The exact factor increase depends primarily on the data types that are in use. For example, Sorted Sets use significantly more memory than Sets, even though both data structures require similar amounts of space when serialized to disk.

This is expected behaviour, given that Redis optimizes heavily both read and write performance.

Note that optimizations are continually being made to reduce the amount of memory required to represent each of the data types in memory.

Problems

Redis exhibits the following issues with persistence:

  • Most save operations require additional memory to complete successfully (as previously discussed). Depending on the size of the dataset, the frequency of writes, and the amount of RAM you are comfortable reserving, this may or may not be an issue.

  • Redis persistence requires extremely heavy I/O usage. This is discussed in detailhere. Also see Salvatore’s response.

  • In some cases, asynchronous saves can block the server for lengthy periods of time. See this post on the mailing list for an interesting discussion.

Although the issues with Redis persistence are hard problems to solve, the issues are beginning to be discussed at length. We should continue to see improvements in this area.

分享到:
评论

相关推荐

    Redis入门指南(第2版)电子书

    《Redis入门指南(第2版)》是一本Redis的入门指导书籍,以通俗易懂的方式介绍了Redis基础与实践方面的知识,包括历史与特性、在开发和生产环境中部署运行Redis、数据类型与命令、使用Redis实现队列、事务、复制、...

    Redis-Essentials.pdf

    Redis is the most popular in-memory key-value data store. It is very lightweight and its data types give it an edge over other competitors. If you need an in-memory database or a high-performance ...

    Redis Essentials

    Redis Essentials is a fast-paced guide that teaches the fundamentals on data types, explains how to manage data through commands, and shares experiences from big players in the industry. We start off...

    基于C#的NewLife.Redis高性能Redis协议封装设计源码

    本项目是基于C#的NewLife.Redis高性能Redis协议封装设计源码,包含119个文件,其中包括95个C#源文件、6个csproj文件、3个YAML文件、2个PNG图片文件、2个pubxml文件、1个Editorconfig文件、1个gitignore文件、1个eddx...

    Redis入门指南(第2版)

    本书是一本Redis入门指导书籍,以通俗易懂的方式介绍了Redis基础与实践方面的知识,包括历史特性、在开发和生产环境中部署运行Redis、数据类型与命令、使用Redis实现队列、事务、复制、管道、持久化、优化Redis存储...

    分布式实现redis锁 出现错误Could not get a resource from the pool

    NULL 博文链接:https://sichen84.iteye.com/blog/2419876

    A SpringBoot project based on Redis from nowcoder..zip

    A SpringBoot project based on Redis from nowcoder.

    Redis in Action

    You'll begin by getting Redis set up properly and then exploring the key-value model. Then, you'll dive into real use cases including simple caching, distributed ad targeting, and more. You'll learn ...

    redis入门指南2

    《Redis入门指南(第2版)》是一本Redis的入门指导书籍,以通俗易懂的方式介绍了Redis基础与实践方面的知识,包括历史与特性、在开发和生产环境中部署运行Redis、数据类型与命令、使用Redis实现队列、事务、复制、...

    Spring+Struts2+hibernate+Redis整合

    在IT行业中,SSH(Spring、Struts2、Hibernate)是一个经典的Java Web开发框架组合,而Redis则是一个高性能的键值存储系统,常用于缓存和数据持久化。将SSH与Redis整合,可以提升应用程序的性能和响应速度。下面将...

    redis配置文件redis.conf

    redis配置文件redis.conf

    windows 64 redis3,2

    2. **Windows 64 位支持**: 在 Windows 操作系统上运行 Redis,需要一个专门为该平台编译的二进制版本。Redis 3.2 提供了对 Windows 64 位系统的支持,使得在 Windows 上部署和管理 Redis 变得更为便捷。用户可以...

    redis2json:以JSON格式导出Redis数据

    redis2json 这是一个快速而肮脏的脚本,它将redis转储到RAM中并打印出相应的JSON。用途您可以通过gzip通过管道传输输出并将其发送到文件,以进行快速的非redis数据备份或导出/迁移到另一个系统。 例如: ./redis_to_...

    Windows版 Redis 5.0.14

    2. **Windows 安装与配置**: - `redis.windows-service.conf` 和 `redis.windows.conf`是Redis在Windows上的配置文件。前者用于将Redis作为服务运行,后者则是标准的配置文件。 - `redis-start.bat`是一个批处理...

    redis-5.0.3 redis-5.0.4 redis-5.0.5

    redis-5.0.3 redis-5.0.4 redis-5.0.5

    Redis两主部署

    Redis 两主部署高可用性解决方案 Redis 作为一个高性能的 NoSQL 数据库,广泛应用于各种行业的数据存储和缓存中。然而,Redis 的高可用性是企业级应用的关键所在。因此,本文将详细介绍 Redis 两主部署的实现方案,...

    Redis Desktop Manager 2022.2.0.0 For Windows

    1.Redis 图形化管理工具 2.Redis Desktop Manager 2022.2 3.数据库工具

Global site tag (gtag.js) - Google Analytics