`
yvonxiao
  • 浏览: 77858 次
  • 性别: Icon_minigender_1
  • 来自: 上海
文章分类
社区版块
存档分类
最新评论

基于kryo序列化方案的memcached-session-manager多memcached节点配置

阅读更多

更新:

现在做分布式解决方案一般都是基于cookie不开启HttpSession来实现的,当时是因为单机系统已经开发好了后面为了方便水平扩展而选择了memcached-session-manager,这几天碰巧又用到了这个,发现最新的稳定版本1.8.3已经很方便的可以使用kryo来做序列化了,这里贴一个我现在用的单机Memcached的配置

 

<Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
    memcachedNodes="n1:127.0.0.1:11211"
    sticky="false"
    lockingMode="auto"
    memcachedProtocol="binary"
    sessionBackupAsync="false"
    requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"
    transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"
    />   

 

新版本不需要以前的一些比如cglib-nodep-2.2.jar等包了,具体可自行参考官网的配置

 

另外说一句,本来是打算用tomcat-redis-session-manager的,后来发现这货实现的方案还是有bug的,比如在spring mvc中使用RedirectAttributes.addFlashAttribute会发现这方法无效,换了它官方说明的Persistence Policies都不行,也不打算替他修正了,直接换久经考验的方式。

----------------------------------------------------------分界线-----------------------------------------------------------------

 

  上次基于Java IO的序列化方案配置了memcached-session-manager,但是性能不好,现在先简单配置成基于kryo的

<Context path="/mobilemail" docBase="D:\webapp\WebRoot" reloadable="true">
<Manager 
className= "de.javakaffee.web.msm.MemcachedBackupSessionManager" 
memcachedNodes= "n1:192.168.112.1:11211,n2:192.168.112.2:11211" 
sticky="false" 
lockingMode="auto" 
requestUriIgnorePattern= ".*\.(png|gif|jpg|css|js)$"  
sessionBackupAsync= "false"
memcachedProtocol="binary" 
copyCollectionsForSerialization="true" 
transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"  
customConverter="org.claros.commons.CustomKryoRegistration"
 />
</Context>

 

 注意customConverter配置的com.test.serializer.CustomKryoRegistration这个类是自己写的类,是用来注册一些特殊类型的,同时自己负责对这些类型的序列化,可以放在项目里,也可以单独打个jar包,把以后遇到的所有复杂类型都在这里即可。项目里遇到的kryo没有注册到的类就是session里的java.util.concurrent.ConcurrentHashMap这个类,反序列化的时候本来可以做得更高效些,这里只是为了演示,直接用kryo自带的MapSerializer来反序列化

 

package com.test.serializer;
import java.util.concurrent.ConcurrentHashMap;

import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryo.serialize.MapSerializer;

import de.javakaffee.web.msm.serializer.kryo.KryoCustomization;

public class CustomKryoRegistration implements KryoCustomization {
	public void customize(Kryo kryo) {
		kryo.register(ConcurrentHashMap.class, new MapSerializer(kryo));
	}
}

 

 

 

感兴趣的同学请参见msm-kryo-serializer-1.4.0.jar注册wicket的MiniMap的流程(de.javakaffee.web.msm.serializer.kryo.WicketMiniMapRegistration里对MiniMap使用kryo-serializers-0.8.jar中的de.javakaffee.kryoserializers.wicket.MiniMapSerializer 单独处理)
 
要使用kryo进行序列化需额外添加kryo的jar包,我使用的是kryo-1.04,所需要的jar包在附件里。添加依赖的时候请注意不要引起jar包冲突,比较常见的是使用cglib和所依赖的asm的冲突,我的解决办法是留下kryo所需要的asm-3.3.1.jar,删去其他版本的asm,再将原先项目里的cglib换成cglib-nodep-2.2.jar。

 

分享到:
评论
3 楼 godo121 2012-06-22  
cheng888qi 写道
你好!
最近也在配置这个但是遇到了问题,web框架中用到了spring security,
我试过了像LZ这样配置,但是 还是出现
Mar 23, 2011 6:24:35 PM de.javakaffee.web.msm.MemcachedBackupSessionManager loadFromMemcached
WARNING: Could not load session with id C9593682F286B6054292A348B27F4CE7-n1 from memcached.
com.esotericsoftware.kryo.SerializationException: Unable to deserialize object of type: java.util.concurrent.ConcurrentHashMap
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:584)
        at com.esotericsoftware.kryo.ObjectBuffer.readObject(ObjectBuffer.java:209)
        at de.javakaffee.web.msm.serializer.kryo.KryoTranscoder.deserializeAttributes(KryoTranscoder.java:257)
        at de.javakaffee.web.msm.TranscoderService.deserializeAttributes(TranscoderService.java:157)
        at de.javakaffee.web.msm.TranscoderService.deserialize(TranscoderService.java:114)
        at de.javakaffee.web.msm.MemcachedBackupSessionManager.loadFromMemcached(MemcachedBackupSessionManager.java:929)
        at de.javakaffee.web.msm.MemcachedBackupSessionManager.findSession(MemcachedBackupSessionManager.java:519)
        at org.apache.catalina.connector.Request.isRequestedSessionIdValid(Request.java:2138)
        at org.apache.catalina.connector.CoyoteAdapter.parseSessionCookiesId(CoyoteAdapter.java:592)
        at org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:509)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:291)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
        at java.lang.Thread.run(Thread.java:662)
Caused by: com.esotericsoftware.kryo.SerializationException: Unable to deserialize object of type: org.springframework.security.core.context.SecurityContextImpl
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:562)
        at com.esotericsoftware.kryo.serialize.MapSerializer.readObjectData(MapSerializer.java:129)
        at com.esotericsoftware.kryo.Serializer.readObject(Serializer.java:61)
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:580)
        ... 14 more
Caused by: com.esotericsoftware.kryo.SerializationException: Serialization trace:
authorities (com.fz.cloudsync.security.user.SecurityUser)
principal (org.springframework.security.authentication.UsernamePasswordAuthenticationToken)
authentication (org.springframework.security.core.context.SecurityContextImpl)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:230)
        at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.readObjectData(ReferenceFieldSerializer.java:81)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:212)
        at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.readObjectData(ReferenceFieldSerializer.java:81)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:212)
        at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.readObjectData(ReferenceFieldSerializer.java:81)
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:557)
        ... 17 more
Caused by: java.lang.ClassCastException: org.springframework.security.core.authority.GrantedAuthorityImpl cannot be cast to java.lang.Comparable
        at java.util.TreeMap.put(TreeMap.java:542)
        at java.util.TreeSet.add(TreeSet.java:238)
        at com.esotericsoftware.kryo.serialize.CollectionSerializer.readObjectData(CollectionSerializer.java:113)
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:557)
        at de.javakaffee.kryoserializers.UnmodifiableCollectionsSerializer.readObjectData(UnmodifiableCollectionsSerializer.java:84)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:212)
        ... 23 more

能帮我解决一下吗?谢谢!

http://www.oschina.net/question/3270_20436
2 楼 yvonxiao 2011-04-16  
cheng888qi 写道
你好!
最近也在配置这个但是遇到了问题,web框架中用到了spring security,
我试过了像LZ这样配置,但是 还是出现
Mar 23, 2011 6:24:35 PM de.javakaffee.web.msm.MemcachedBackupSessionManager loadFromMemcached
WARNING: Could not load session with id C9593682F286B6054292A348B27F4CE7-n1 from memcached.
com.esotericsoftware.kryo.SerializationException: Unable to deserialize object of type: java.util.concurrent.ConcurrentHashMap
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:584)
        at com.esotericsoftware.kryo.ObjectBuffer.readObject(ObjectBuffer.java:209)
        at de.javakaffee.web.msm.serializer.kryo.KryoTranscoder.deserializeAttributes(KryoTranscoder.java:257)
        at de.javakaffee.web.msm.TranscoderService.deserializeAttributes(TranscoderService.java:157)
        at de.javakaffee.web.msm.TranscoderService.deserialize(TranscoderService.java:114)
        at de.javakaffee.web.msm.MemcachedBackupSessionManager.loadFromMemcached(MemcachedBackupSessionManager.java:929)
        at de.javakaffee.web.msm.MemcachedBackupSessionManager.findSession(MemcachedBackupSessionManager.java:519)
        at org.apache.catalina.connector.Request.isRequestedSessionIdValid(Request.java:2138)
        at org.apache.catalina.connector.CoyoteAdapter.parseSessionCookiesId(CoyoteAdapter.java:592)
        at org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:509)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:291)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
        at java.lang.Thread.run(Thread.java:662)
Caused by: com.esotericsoftware.kryo.SerializationException: Unable to deserialize object of type: org.springframework.security.core.context.SecurityContextImpl
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:562)
        at com.esotericsoftware.kryo.serialize.MapSerializer.readObjectData(MapSerializer.java:129)
        at com.esotericsoftware.kryo.Serializer.readObject(Serializer.java:61)
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:580)
        ... 14 more
Caused by: com.esotericsoftware.kryo.SerializationException: Serialization trace:
authorities (com.fz.cloudsync.security.user.SecurityUser)
principal (org.springframework.security.authentication.UsernamePasswordAuthenticationToken)
authentication (org.springframework.security.core.context.SecurityContextImpl)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:230)
        at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.readObjectData(ReferenceFieldSerializer.java:81)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:212)
        at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.readObjectData(ReferenceFieldSerializer.java:81)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:212)
        at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.readObjectData(ReferenceFieldSerializer.java:81)
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:557)
        ... 17 more
Caused by: java.lang.ClassCastException: org.springframework.security.core.authority.GrantedAuthorityImpl cannot be cast to java.lang.Comparable
        at java.util.TreeMap.put(TreeMap.java:542)
        at java.util.TreeSet.add(TreeSet.java:238)
        at com.esotericsoftware.kryo.serialize.CollectionSerializer.readObjectData(CollectionSerializer.java:113)
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:557)
        at de.javakaffee.kryoserializers.UnmodifiableCollectionsSerializer.readObjectData(UnmodifiableCollectionsSerializer.java:84)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:212)
        ... 23 more

能帮我解决一下吗?谢谢!

你配置了customConverter么,这个要自己去写反序列化的类的,java.lang.ClassCastException:org.springframework.security.core.authority.GrantedAuthorityImpl cannot be cast to java.lang.Comparable,我给的例子是做个例子来演示的,你要根据你的实际情况自己处理
1 楼 cheng888qi 2011-03-24  
你好!
最近也在配置这个但是遇到了问题,web框架中用到了spring security,
我试过了像LZ这样配置,但是 还是出现
Mar 23, 2011 6:24:35 PM de.javakaffee.web.msm.MemcachedBackupSessionManager loadFromMemcached
WARNING: Could not load session with id C9593682F286B6054292A348B27F4CE7-n1 from memcached.
com.esotericsoftware.kryo.SerializationException: Unable to deserialize object of type: java.util.concurrent.ConcurrentHashMap
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:584)
        at com.esotericsoftware.kryo.ObjectBuffer.readObject(ObjectBuffer.java:209)
        at de.javakaffee.web.msm.serializer.kryo.KryoTranscoder.deserializeAttributes(KryoTranscoder.java:257)
        at de.javakaffee.web.msm.TranscoderService.deserializeAttributes(TranscoderService.java:157)
        at de.javakaffee.web.msm.TranscoderService.deserialize(TranscoderService.java:114)
        at de.javakaffee.web.msm.MemcachedBackupSessionManager.loadFromMemcached(MemcachedBackupSessionManager.java:929)
        at de.javakaffee.web.msm.MemcachedBackupSessionManager.findSession(MemcachedBackupSessionManager.java:519)
        at org.apache.catalina.connector.Request.isRequestedSessionIdValid(Request.java:2138)
        at org.apache.catalina.connector.CoyoteAdapter.parseSessionCookiesId(CoyoteAdapter.java:592)
        at org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:509)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:291)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
        at java.lang.Thread.run(Thread.java:662)
Caused by: com.esotericsoftware.kryo.SerializationException: Unable to deserialize object of type: org.springframework.security.core.context.SecurityContextImpl
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:562)
        at com.esotericsoftware.kryo.serialize.MapSerializer.readObjectData(MapSerializer.java:129)
        at com.esotericsoftware.kryo.Serializer.readObject(Serializer.java:61)
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:580)
        ... 14 more
Caused by: com.esotericsoftware.kryo.SerializationException: Serialization trace:
authorities (com.fz.cloudsync.security.user.SecurityUser)
principal (org.springframework.security.authentication.UsernamePasswordAuthenticationToken)
authentication (org.springframework.security.core.context.SecurityContextImpl)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:230)
        at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.readObjectData(ReferenceFieldSerializer.java:81)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:212)
        at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.readObjectData(ReferenceFieldSerializer.java:81)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:212)
        at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.readObjectData(ReferenceFieldSerializer.java:81)
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:557)
        ... 17 more
Caused by: java.lang.ClassCastException: org.springframework.security.core.authority.GrantedAuthorityImpl cannot be cast to java.lang.Comparable
        at java.util.TreeMap.put(TreeMap.java:542)
        at java.util.TreeSet.add(TreeSet.java:238)
        at com.esotericsoftware.kryo.serialize.CollectionSerializer.readObjectData(CollectionSerializer.java:113)
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:557)
        at de.javakaffee.kryoserializers.UnmodifiableCollectionsSerializer.readObjectData(UnmodifiableCollectionsSerializer.java:84)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:212)
        ... 23 more

能帮我解决一下吗?谢谢!

相关推荐

    session共享 memcached-session-manager 1.9.6 jar

    标签中的"msn-kryo"是指Kryo序列化库,它是`memcached-session-manager`可选的序列化机制之一。Kryo以其高效的序列化和反序列化速度而闻名,可以显著提升Memcached操作session的速度。而"高性能缓存"则强调了...

    Memcached-session-manager所需最新jar包-tomcat7-kryo序列

    【标题】:“Memcached-session-manager所需最新jar包-tomcat7-kryo序列”指的是在Tomcat7中使用Memcached作为会话管理器时,需要的特定版本的jar包,特别是与Kryo序列化相关的组件。 【描述】:“Memcached-...

    memcached-session-manager

    1. **分布式会话管理**:在高并发的Web环境中,单一服务器的会话管理可能成为瓶颈,memcached-session-manager 提供了解决方案,它将用户的会话数据分布存储在多个 Memcached 服务器上,实现了会话的跨服务器共享,...

    tomcat-7.0.54配合使用的memcached-session-manager1.6.1的整套jar包

    transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory" // 使用Kryo序列化器 /&gt; ``` 在这个例子中,我们选择了KryoTranscoderFactory作为默认的序列化器,因为它的性能通常优于...

    tomcat-session序列化(tomcat-kryo序列化)

    tomcat kryo序列化,msm-memcached-session-manager-tc7-1.8.2

    memcached-session-manager集成包

    4. **msm-javolution-serializer-1.8.3.jar** 和 **msm-kryo-serializer-1.8.3.jar**:这两个JAR文件提供了不同的序列化策略,javolution序列化器和kryo序列化器,用户可以根据需求选择合适的序列化方式。...

    Memcached_Session_Manager jar

    总结起来,Memcached_Session_Manager jar是用于Tomcat集群环境的一个解决方案,它借助Memcached作为中央会话存储,通过javolution和kryo提高序列化效率,确保在分布式环境中正确、高效地管理用户会话。使用这个组件...

    memcached-session-manager tomcat7

    《基于Tomcat7的Memcached会话...结合`spymemcached`、Kryo序列化库和其他相关组件,我们可以构建出一个高度可扩展和高可用的Web服务环境。然而,配置过程中需要注意版本兼容性和性能优化,以确保最佳的系统运行效果。

    tomcat8+memcached-session的连接1.9.6版本jar包

    总结起来,"tomcat8+memcached-session的连接1.9.6版本jar包"是一个解决分布式环境中会话管理问题的解决方案。它通过高效的序列化库Kryo和专门为Tomcat8设计的适配器,实现了与Memcached的无缝连接,有效提升了多...

    apache-tomcat-7.0.47-memcached-各种序列化策略-session共享

    4. **apache-tomcat-7.0.47-memcached-kryo-session共享.rar**:Kryo是一个用于Java的快速、高效的对象图形序列化框架。它在速度和空间效率方面表现出色,尤其适合处理大量数据的场景。 5. **apache-tomcat-7.0.47-...

    tomcat8+memcached session共享

    5. `msm-kryo-serializer-1.8.3.jar`:这是Kryo序列化器的扩展,专门为memcached-session-manager提供支持。 6. `reflectasm-1.09.jar`:ReflectASM是一个用于反射的轻量级库,提高了反射操作的性能,可能在Kryo序列...

    memcached-session-manager-1.9.3-tomcat6-all

    压缩包里是我自己下载源码测试打包的memcached-session-manager-1.9.3 for tomcat6相关jar文件,序列化采用的是kryo-serializer。我的运行环境是:JDK1.6+TOMCAT6.0.45,MSM加Tomcat有一个特别大的坑是如果Tomcat版本是...

    memcached共享session需要的jar包

    6. **msm-kryo-serializer-1.6.5.jar**:这是memcached-session-manager的一个扩展,专门为Kryo序列化提供支持。 7. **reflectasm-1.01.jar**:ReflectASM是一个轻量级的反射库,比Java标准的反射API更快,它可能被...

    memcache-session序列化

    tomcat kryo memcache session序列化 msm-memcached-session-manager-1.8.2

    memcached tomcat session 共享所需jar

    这个jar包提供了一个基于Kryo的序列化策略,供Memcached Session Manager选择。 7. **memcached-session-manager-tc7-1.5.1.jar**:这个jar是针对Tomcat 7的适配器,确保Memcached Session Manager能与Tomcat 7版本...

    Tomcat +memcached 依赖jar包

    5. **msm-kryo-serializer-1.5.1.jar**:这个jar包包含了基于Kryo的序列化器,与memcached-session-manager一起工作,提供另一种序列化session对象到memcached的方式。由于Kryo的高性能,这通常会比默认的Java序列化...

    memcached session manager 1.9.6 适用于 tomcat 8 各版本 支持kryo等多种序列化方式

    6. **msm-kryo-3.0.3.jar**: Kryo的库文件,用于实现Kryo序列化。 7. **msm-memcached-session-manager-1.9.6.jar**: 主要的memcached session manager实现。 8. **msm-flexjson-3.3.jar**: FlexJSON库,用于JSON...

    memcached各种序列化策略之session共享

    标题提到的"memcached各种序列化策略之session共享",意味着本文将探讨如何在Memcached中使用不同的序列化方法来共享Web应用中的session数据。Memcached是一个高性能、分布式内存对象缓存系统,常用于减轻数据库负载...

    memcach-session-manager(MSM)所需jar包,TOMCAT7

    1. `memcached-session-manager.jar`:MSM的主要实现,负责与Memcached服务器的交互和会话管理。 2. `tomee-memcached.jar` 或 `tomcat-memcached.jar`:这是针对Tomcat的适配器,使得MSM能够集成到Tomcat7中。 3. `...

Global site tag (gtag.js) - Google Analytics