扩展spring boot cache实现redis一二级分布式缓存

sundoctor

浏览: 325739 次
性别:
来自: 北京

最近访客更多访客>>

tcytree

ricki

为梦想起航

iiyangyiii

博主相关

博客

微博

相册

留言

关于我

文章分类

社区版块

存档分类

博客分类：

spring boot cache redis java

系统为了提高数据访问速度，先将数据加载到redis缓存中，但是每次从缓存获取数据，要通过网络访问才能获取，效率还是不够逆天快。如果访问量很大，并发很高，性能不够快不说，还容易造成reids负载过高，redis的主机出现各种物理故障。因此，可以在redis前增加本地一级缓存，本地一级缓存和系统应用在同一个JVM内，这样速度最快，redis退居二线当作二级缓存。每次请求先从一级缓存读取数据，一级缓存没有数据，再从二级缓存读取，并同步到一级缓存里，通过redis的消息发布订阅通知其他client机器更新缓存。这同CPU的一级缓存，二级缓存是一个道理。

本文并不涉及spring boot cache的详细使用介绍，需要熟悉spring boot cache基本使用。对于spring boot cache详细使用介绍请上度娘。扩展比较简单，闲话少说，直接上代码。

pom.xml，加入spring boot cache依赖，本地一级缓存使用ehcache3，二级缓存使用redis

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.0.0.M7</version>
    </parent>
    <groupId>springboot</groupId>
    <artifactId>springboot-2level-cache</artifactId>
    <version>1.0-SNAPSHOT</version>

    <dependencies>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-cache</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-redis</artifactId>
        </dependency>
        <dependency>
            <groupId>redis.clients</groupId>
            <artifactId>jedis</artifactId>
        </dependency>
        <dependency>
            <groupId>javax.cache</groupId>
            <artifactId>cache-api</artifactId>
        </dependency>
        <dependency>
            <groupId>org.ehcache</groupId>
            <artifactId>ehcache</artifactId>
        </dependency>

        <!-- Only used to expose cache metrics -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>

        <!-- Test -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
    <repositories>
        <repository>
            <id>spring-milestones</id>
            <name>Spring Milestones</name>
            <url>https://repo.spring.io/libs-milestone</url>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </repository>
    </repositories>

</project>

appliaction.properties里配置属性

spring.cache.type=redis
spring.ext.cache.name=countries
spring.ext.cache.redis.topic=cache

本地一级缓存使用ehcache3，ehcache3.xml配置

<config xmlns='http://www.ehcache.org/v3'
		xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
		xmlns:jsr107="http://www.ehcache.org/v3/jsr107"
		xsi:schemaLocation="http://www.ehcache.org/v3 http://www.ehcache.org/schema/ehcache-core-3.0.xsd
							http://www.ehcache.org/v3/jsr107 http://www.ehcache.org/schema/ehcache-107-ext-3.0.xsd">
	<cache alias="countries">
		<expiry>
			<ttl unit="seconds">600</ttl>
		</expiry>
		<heap unit="entries">200</heap>
		<jsr107:mbeans enable-statistics="true"/>
	</cache>

</config>

创建LocalRedisCache ，继承RedisCache，这里是实现redis一二级分布式缓存的核心，重载RedisCache的get、put、evict、clean等方法，增加本地一级缓存的读写，增加pub方法，通过redis发布缓存更新消息通知其他 redis clent 更新缓存，增加sub方法，获取缓存更新消息更新缓存。

package org.springframework.data.redis.cache;

import org.springframework.cache.Cache;
import org.springframework.data.redis.core.RedisOperations;
import org.springframework.util.Assert;

import java.util.concurrent.Callable;

/**
 * extends RedisCache，增加本地一级缓存，redis作为二级缓存
 */
public class LocalRedisCache extends RedisCache {

    private final Cache localCache;//本地一级缓存
    private final RedisOperations redisOperations;//配合topicName，发布缓存更新消息
    private final String topicName;//redis topic ，发布缓存更新消息通知其他client更新缓存

    /**
     * Create new {@link RedisCache}.
     *
     * @param name            must not be {@literal null}.
     * @param cacheWriter     must not be {@literal null}.
     * @param cacheConfig     must not be {@literal null}.
     * @param localCache      must not be {@literal null}.
     * @param redisOperations must not be {@literal null}.
     * @param topicName       must not be {@literal null}.
     */
    protected LocalRedisCache(String name, RedisCacheWriter cacheWriter, RedisCacheConfiguration cacheConfig,
                              Cache localCache, RedisOperations redisOperations, String topicName) {

        super(name, cacheWriter, cacheConfig);

        Assert.notNull(localCache, "localCache must not be null!");
        Assert.notNull(redisOperations, "redisOperations must not be null!");
        Assert.hasText(topicName, "topicName must not be empty!");

        this.localCache = localCache;
        this.redisOperations = redisOperations;
        this.topicName = topicName;
    }

    @Override
    public synchronized <T> T get(Object key, Callable<T> valueLoader) {
        //先读取本地一级缓存
        T value = localCache.get(key, valueLoader);
        if (value == null) {
            //本地一级缓存不存在，读取redis二级缓存
            value = super.get(key, valueLoader);
            if (value != null) {
                //redis二级缓存存在，存入本地一级缓存
                localCache.put(key, value);
                //发布缓存更新消息通知其他client更新缓存
                pub(new UpdateMessage(key, value, UpdateMessage.Type.PUT));
            }
        }
        return value;
    }

    @Override
    public void put(Object key, Object value) {
        super.put(key, value);
        localCache.put(key, value);
        pub(new UpdateMessage(key, value, UpdateMessage.Type.PUT));
    }

    @Override
    public ValueWrapper putIfAbsent(Object key, Object value) {
        ValueWrapper vw1 = localCache.putIfAbsent(key, value);
        ValueWrapper vw2 = super.putIfAbsent(key, value);

        pub(new UpdateMessage(key, value, UpdateMessage.Type.PUTIFABSENT));

        return vw1 == null ? vw2 : vw1;
    }

    @Override
    public void evict(Object key) {
        localCache.evict(key);
        super.evict(key);
        pub(new UpdateMessage(key, UpdateMessage.Type.REMOVE));
    }

    @Override
    public void clear() {
        localCache.clear();
        super.clear();
        pub(new UpdateMessage(UpdateMessage.Type.CLEAN));
    }

    @Override
    public ValueWrapper get(Object key) {
        ValueWrapper valueWrapper = localCache.get(key);
        if (valueWrapper == null) {
            valueWrapper = super.get(key);
            if (valueWrapper != null) {
                localCache.put(key, valueWrapper.get());
                pub(new UpdateMessage(key, valueWrapper.get(), UpdateMessage.Type.PUT));
            }
        }
        return valueWrapper;
    }

    @Override
    public <T> T get(Object key, Class<T> type) {
        T value = localCache.get(key, type);
        if (value == null) {
            value = super.get(key, type);
            if (value != null) {
                localCache.put(key, value);
                pub(new UpdateMessage(key, value, UpdateMessage.Type.PUT));
            }
        }
        return value;
    }

    /**
     * 更新缓存
     *
     * @param updateMessage
     */
    public void sub(final UpdateMessage updateMessage) {

        if (updateMessage.getType() == UpdateMessage.Type.CLEAN) {
            //清除所有缓存
            localCache.clear();
            super.clear();
        } else if (updateMessage.getType() == UpdateMessage.Type.PUT) {
            //更新缓存
            localCache.put(updateMessage.getKey(), updateMessage.getValue());
            super.put(updateMessage.getKey(), updateMessage.getValue());
        } else if (updateMessage.getType() == UpdateMessage.Type.PUTIFABSENT) {
            //更新缓存
            localCache.putIfAbsent(updateMessage.getKey(), updateMessage.getValue());
            super.putIfAbsent(updateMessage.getKey(), updateMessage.getValue());
        } else if (updateMessage.getType() == UpdateMessage.Type.REMOVE) {
            //删除缓存
            localCache.evict(updateMessage.getKey());
            super.evict(updateMessage.getKey());
        }
    }

    /**
     * 通知其他 redis clent 更新缓存
     *
     * @param message
     */
    private void pub(final UpdateMessage message) {
        this.redisOperations.convertAndSend(topicName, message);
    }

}

创建新的缓存管理器，命名为LocalRedisCacheManager，继承了Spring Boot的RedisCacheManager，重载createRedisCache方法。

package org.springframework.data.redis.cache;

import org.springframework.cache.Cache;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.core.RedisOperations;
import org.springframework.util.Assert;

import java.util.Map;

public class LocalRedisCacheManager extends RedisCacheManager {

    private final RedisConnectionFactory connectionFactory;
    private final Cache localCache;
    private final RedisOperations redisOperations;
    private final String topicName;

    public LocalRedisCacheManager(RedisCacheWriter cacheWriter, RedisCacheConfiguration defaultCacheConfiguration,
                                  RedisConnectionFactory connectionFactory, Cache localCache, RedisOperations redisOperations,
                                  String topicName) {
        super(cacheWriter, defaultCacheConfiguration);


        this.connectionFactory = connectionFactory;
        this.localCache = localCache;
        this.redisOperations = redisOperations;
        this.topicName = topicName;

        check();
    }

    public LocalRedisCacheManager(RedisCacheWriter cacheWriter, RedisCacheConfiguration defaultCacheConfiguration,
                                  RedisConnectionFactory connectionFactory, Cache localCache,
                                  RedisOperations redisOperations, String topicName, String... initialCacheNames) {
        super(cacheWriter, defaultCacheConfiguration, initialCacheNames);


        this.connectionFactory = connectionFactory;
        this.localCache = localCache;
        this.redisOperations = redisOperations;
        this.topicName = topicName;

        check();
    }

    public LocalRedisCacheManager(RedisCacheWriter cacheWriter, RedisCacheConfiguration defaultCacheConfiguration,
                                  Map<String, RedisCacheConfiguration> initialCacheConfigurations,
                                  RedisConnectionFactory connectionFactory, Cache localCache,
                                  RedisOperations redisOperations, String topicName) {
        super(cacheWriter, defaultCacheConfiguration, initialCacheConfigurations);

        this.connectionFactory = connectionFactory;
        this.localCache = localCache;
        this.redisOperations = redisOperations;
        this.topicName = topicName;

        check();
    }

    @Override
    protected RedisCache createRedisCache(String name, RedisCacheConfiguration cacheConfig) {
        return new LocalRedisCache(name, new DefaultRedisCacheWriter(connectionFactory),
                cacheConfig != null ? cacheConfig : RedisCacheConfiguration.defaultCacheConfig(),
                localCache, redisOperations, topicName);
    }

    public static LocalRedisCacheManager create(RedisConnectionFactory connectionFactory, Cache localCache, RedisOperations redisOperations, String topicName) {

        Assert.notNull(localCache, "localCache must not be null");
        Assert.notNull(connectionFactory, "connectionFactory must not be null");
        Assert.notNull(redisOperations, "redisOperations must not be null");
        Assert.notNull(topicName, "topicName must not be null");

        return new LocalRedisCacheManager(new DefaultRedisCacheWriter(connectionFactory),
                RedisCacheConfiguration.defaultCacheConfig(), connectionFactory, localCache, redisOperations, topicName);
    }


    private void check() {
        Assert.notNull(localCache, "localCache must not be null");
        Assert.notNull(connectionFactory, "connectionFactory must not be null");
        Assert.notNull(redisOperations, "redisOperations must not be null");
        Assert.notNull(topicName, "topicName must not be null");
    }


}

缓存配置实现，需要使用注解 @EnableCaching 打开缓存功能

package org.springframework.data.redis.cache;

import org.springframework.beans.factory.annotation.Value;
import org.springframework.cache.CacheManager;
import org.springframework.cache.annotation.EnableCaching;
import org.springframework.cache.jcache.JCacheCacheManager;
import org.springframework.cache.jcache.JCacheManagerFactoryBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.core.io.ClassPathResource;
import org.springframework.core.io.Resource;
import org.springframework.data.redis.connection.Message;
import org.springframework.data.redis.connection.MessageListener;
import org.springframework.data.redis.connection.jedis.JedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.listener.PatternTopic;
import org.springframework.data.redis.listener.RedisMessageListenerContainer;
import org.springframework.data.redis.listener.adapter.MessageListenerAdapter;

import java.io.IOException;

@Configuration
@EnableCaching
public class CacheConfig {

    @Value("${spring.ext.cache.name:countries}")
    private String localCacheName;

    @Value("${spring.ext.cache.redis.topic:cache}")
    private String topicName;

    @Bean
    public CacheManager jCacheCacheManager() {
        return new JCacheCacheManager(jCacheManagerFactoryBean().getObject());
    }

    @Bean
    public JCacheManagerFactoryBean jCacheManagerFactoryBean() {
        JCacheManagerFactoryBean jCacheManagerFactoryBean = new JCacheManagerFactoryBean();
        Resource resource = new ClassPathResource("ehcache3.xml");
        try {
            jCacheManagerFactoryBean.setCacheManagerUri(resource.getURI());
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
        return jCacheManagerFactoryBean;
    }

    @Bean
    public JedisConnectionFactory jedisConnectionFactory() {
        JedisConnectionFactory factory = new JedisConnectionFactory();
        return factory;
    }

    @Bean
    public RedisTemplate redisTemplate(JedisConnectionFactory jedisConnectionFactory) {
        RedisTemplate redisTemplate = new RedisTemplate();
        redisTemplate.setConnectionFactory(jedisConnectionFactory);
        return redisTemplate;
    }

    @Primary
    @Bean
    public RedisCacheManager redisCacheManager(JedisConnectionFactory jedisConnectionFactory,
                                               RedisTemplate redisTemplate) {
        RedisCacheManager cacheManager = LocalRedisCacheManager.create(jedisConnectionFactory,
                jCacheCacheManager().getCache(localCacheName), redisTemplate, topicName);
        return cacheManager;
    }


    @Bean
    public RedisMessageListenerContainer container(JedisConnectionFactory jedisConnectionFactory,
                                                   MessageListenerAdapter listenerAdapter) {

        RedisMessageListenerContainer container = new RedisMessageListenerContainer();
        container.setConnectionFactory(jedisConnectionFactory);
        container.addMessageListener(listenerAdapter, new PatternTopic(topicName));

        return container;
    }

    @Bean
    public MessageListenerAdapter listenerAdapter(LocalRedisCacheManager cacheManager, RedisTemplate redisTemplate) {
        return new MessageListenerAdapter(new MessageListener() {
            @Override
            public void onMessage(Message message, byte[] pattern) {
                byte[] channel = message.getChannel();
                byte[] body = message.getBody();

                String cacheName = (String) redisTemplate.getStringSerializer().deserialize(channel);
                LocalRedisCache cache = (LocalRedisCache) cacheManager.getCache(cacheName);
                if (cache == null) {
                    return;
                }

                UpdateMessage updateMessage = (UpdateMessage) redisTemplate.getValueSerializer().deserialize(body);
                cache.sub( updateMessage);
            }
        });
    }

}

到些，实现基本完成，其他相关简单代码，参考附件。

springboot-2level-cache.zip (67.9 KB)
下载次数: 87

分享到：

将博客搬至CSDN | 一致性哈希算法的Java实现

2018-01-27 00:29
浏览 6343
评论(1)
分类:开源软件
查看更多

1 楼无忧何往 2019-01-18

实际用了,发现有点问题;
1.redis发布的message应该带上cache name,这样方便获取;如果是按照文中用topic,也就是channel来获取每次都是一样的
2.构造RedisCacheManager时候应该传入EcacheManager
3.应该是自己实现一个cachemanager然后持有二级缓存(redisCacheManager)和一级缓存(EcacheManager)的引用;这样耦合不紧密

发表评论

您还没有登录,请您登录后再发表评论

最近访客更多访客>>

博主相关

文章分类

社区版块

存档分类

最新评论