一次jboss中部署应用时类版本冲突问题分析、解决过程

去年同事的一个项目在JBOSS中部署时遇到类版本冲突问题，当时协助他处理了此问题，现在将当时处理的过程记录一下，以备以后参考使用。错误日志如下：

2010-12-13 17:06:00,101 ERROR [STDERR] Exception in thread "DefaultQuartzScheduler_Worker-1" 
2010-12-13 17:06:00,101 ERROR [STDERR] java.lang.NoSuchMethodError: org.codehaus.jettison.AbstractXMLStreamWriter.getSerializedAsArrays()Ljava/util/ArrayList;
2010-12-13 17:06:00,101 ERROR [STDERR]  at com.thoughtworks.xstream.io.json.JettisonStaxWriter.startNode(JettisonStaxWriter.java:71)
2010-12-13 17:06:00,101 ERROR [STDERR]  at com.thoughtworks.xstream.io.WriterWrapper.startNode(WriterWrapper.java:33)
2010-12-13 17:06:00,101 ERROR [STDERR]  at com.thoughtworks.xstream.io.path.PathTrackingWriter.startNode(PathTrackingWriter.java:44)
2010-12-13 17:06:00,101 ERROR [STDERR]  at com.thoughtworks.xstream.io.ExtendedHierarchicalStreamWriterHelper.startNode(ExtendedHierarchicalStreamWriterHelper.java:17)
2010-12-13 17:06:00,101 ERROR [STDERR]  at com.thoughtworks.xstream.core.TreeMarshaller.start(TreeMarshaller.java:96)
2010-12-13 17:06:00,101 ERROR [STDERR]  at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.marshal(AbstractTreeMarshallingStrategy.java:38)
2010-12-13 17:06:00,101 ERROR [STDERR]  at com.thoughtworks.xstream.XStream.marshal(XStream.java:837)
2010-12-13 17:06:00,102 ERROR [STDERR]  at com.thoughtworks.xstream.XStream.marshal(XStream.java:826)
2010-12-13 17:06:00,102 ERROR [STDERR]  at com.thoughtworks.xstream.XStream.toXML(XStream.java:801)
2010-12-13 17:06:00,102 ERROR [STDERR]  at com.thoughtworks.xstream.XStream.toXML(XStream.java:789)
2010-12-13 17:06:00,102 ERROR [STDERR]  at org.springframework.batch.core.repository.dao.XStreamExecutionContextStringSerializer.serialize(XStreamExecutionContextStringSerializer.java:43)

    从上面的日志可以看出spring batch 依赖的jettison的AbstractXMLStreamWriter类找不到getSerializedAsArrays(java.util.ArrayList)方法。遇到这个问题，我们很自然的先检查一下xxxx-ear.ear/lib/codehaus.jettison-1.0.1.jar这个jar中的AbstractXMLStreamWriter类是否符合要求，检查结果这个类存在getSerializedAsArrays(java.util.ArrayList)方法。这样也就是说jboss没有加载到ear包中的类。
    下面我们来跟踪一下，看看应用究竟在那个jar里加载了AbstractXMLStreamWriter类。首先我们需要修改一下jboss log4j的配置，这个配置的意思相信大家一看就了解了。

<appender name="UCL" class="org.apache.log4j.FileAppender">
        <param name="File" value="/home/sean/output/logs/user/ucl.log"/>
        <layout class="org.apache.log4j.PatternLayout">
                <param name="ConversionPattern" value="[%r,%c{1},%t] %m%n"/>
        </layout>
    </appender>

    <category name="org.jboss.mx.loading" additivity="false">
        <priority value="TRACE" class="org.jboss.logging.XLevel"></priority>
        <appender-ref ref="UCL"></appender-ref>
    </category>
 

       
然后，我们查看ucl的日志可以看到下列信息

[10141,ClassLoaderUtils,main] Indexed pkg: org.codehaus.jettison, UCL: org.jboss.mx.loading.UnifiedClassLoader3@1fe500a{ url=file:/home/sean/web-deploy/jboss_server/default/deplo
y/jbossws.sar/ ,addedOrder=12}
[10141,ClassLoaderUtils,main] Indexed pkg: org.codehaus.jettison.badgerfish, UCL: org.jboss.mx.loading.UnifiedClassLoader3@1fe500a{ url=file:/home/sean/web-deploy/jboss_server/de
fault/deploy/jbossws.sar/ ,addedOrder=12}
[10142,ClassLoaderUtils,main] Indexed pkg: org.codehaus.jettison.json, UCL: org.jboss.mx.loading.UnifiedClassLoader3@1fe500a{ url=file:/home/sean/web-deploy/jboss_server/default/
deploy/jbossws.sar/ ,addedOrder=12}
[10143,ClassLoaderUtils,main] Indexed pkg: org.codehaus.jettison.mapped, UCL: org.jboss.mx.loading.UnifiedClassLoader3@1fe500a{ url=file:/home/sean/web-deploy/jboss_server/defaul
t/deploy/jbossws.sar/ ,addedOrder=12}
[10144,ClassLoaderUtils,main] Indexed pkg: org.codehaus.jettison.util, UCL: org.jboss.mx.loading.UnifiedClassLoader3@1fe500a{ url=file:/home/sean/web-deploy/jboss_server/default/
deploy/jbossws.sar/ ,addedOrder=12}
[10144,RepositoryClassLoader,main] Added url: file:/home/sean/output/logs/jboss_stout/tmp/deploy/tmp8907418620370263381jettison.jar, to ucl: org.jboss.mx.loading.UnifiedClassLoad
er3@1fe500a{ url=file:/home/sean/web-deploy/jboss_server/default/deploy/jbossws.sar/ ,addedOrder=12}

从上面的日志可以看出有个jettison.jar加到了这个ucl org.jboss.mx.loading.UnifiedClassLoader3@1fe500a中。从日志里还可以看到这个jar存在的路径是/home/sean/web-deploy/jboss_server/default/deploy/jbossws.sar/

再来看一段日志

[19814,ClassLoaderUtils,main] Multiple class loaders found for pkg: org.codehaus.jettison.badgerfish
[19814,ClassLoaderUtils,main] Indexed pkg: org.codehaus.jettison.badgerfish, UCL: org.jboss.mx.loading.UnifiedClassLoader3@16274ea{ url=file:/home/sean/output/logs/jboss_stout/tm
p/deploy/tmp7754042895360577419xxxx-ear.ear ,addedOrder=45}
[19814,ClassLoaderUtils,main] Multiple class loaders found for pkg: org.codehaus.jettison
[19814,ClassLoaderUtils,main] Indexed pkg: org.codehaus.jettison, UCL: org.jboss.mx.loading.UnifiedClassLoader3@16274ea{ url=file:/home/sean/output/logs/jboss_stout/tmp/deploy/tm
p7754042895360577419xxxx-ear.ear ,addedOrder=45}
[19814,ClassLoaderUtils,main] Multiple class loaders found for pkg: org.codehaus.jettison.json
[19814,ClassLoaderUtils,main] Indexed pkg: org.codehaus.jettison.json, UCL: org.jboss.mx.loading.UnifiedClassLoader3@16274ea{ url=file:/home/sean/output/logs/jboss_stout/tmp/depl
oy/tmp7754042895360577419xxxx-ear.ear ,addedOrder=45}
[19815,ClassLoaderUtils,main] Multiple class loaders found for pkg: org.codehaus.jettison.util
[19815,ClassLoaderUtils,main] Indexed pkg: org.codehaus.jettison.util, UCL: org.jboss.mx.loading.UnifiedClassLoader3@16274ea{ url=file:/home/sean/output/logs/jboss_stout/tmp/depl
oy/tmp7754042895360577419xxxx-ear.ear ,addedOrder=45}
[19815,ClassLoaderUtils,main] Multiple class loaders found for pkg: org.codehaus.jettison.mapped
[19815,ClassLoaderUtils,main] Indexed pkg: org.codehaus.jettison.mapped, UCL: org.jboss.mx.loading.UnifiedClassLoader3@16274ea{ url=file:/home/sean/output/logs/jboss_stout/tmp/de
ploy/tmp7754042895360577419xxxx-ear.ear ,addedOrder=45}
[19816,RepositoryClassLoader,main] Added url: file:/home/sean/output/logs/jboss_stout/tmp/deploy/tmp7754042895360577419xxxx-ear.ear-contents/lib/codehaus.jettison-1.0.1.jar, t
o ucl: org.jboss.mx.loading.UnifiedClassLoader3@16274ea{ url=file:/home/sean/output/logs/jboss_stout/tmp/deploy/tmp7754042895360577419xxxx-ear.ear ,addedOrder=45}

从上面的日志可以看到又有一个codehaus.jettison-1.0.1.jar加到了这个ucl org.jboss.mx.loading.UnifiedClassLoader3@16274ea中这个是我们应用自己的jar(xxxx-ear.ear-contents/lib/codehaus.jettison-1.0.1.jar,)
        
在应用加载AbstractXMLStreamWriter类时出现了如下日志

[70806,RepositoryClassLoader,DefaultQuartzScheduler_Worker-9] loadClass org.jboss.mx.loading.UnifiedClassLoader3@1fe500a{ url=file:/home/sean/web-deploy/jboss_server/default/depl
oy/jbossws.sar/ ,addedOrder=12} name=org.codehaus.jettison.AbstractXMLStreamWriter class=class org.codehaus.jettison.AbstractXMLStreamWriter cl=org.jboss.mx.loading.UnifiedClassLo
ader3@1fe500a{ url=file:/home/sean/web-deploy/jboss_server/default/deploy/jbossws.sar/ ,addedOrder=12}

    从这个日志中我们可以看出来AbstractXMLStreamWriter是从org.jboss.mx.loading.UnifiedClassLoader3@1fe500a中加载到的也就是说这个class来自/home/sean/web-deploy/jboss_server/default/deploy/jbossws.sar/下的jettison.jar中，根本就不是应用想要的类。

    为什么会出现上面这样的情况呢？？？
    这就要从jboss的类加载机制说起。请查看我以前写的一篇关于jboss classloader的分析文章

分析了问题的原因之后，我们来看一下如何解决此问题
参考这里http://community.jboss.org/wiki/JBossClassLoadingUseCases的介绍，我一开始是参考Case 3的做法将ear的Repository和应用服务器中的默认的Repository进行隔离。实现如下结构

引用

Because Java2ParentDelegation is turned off by default, the Step (1.1) is never executed,parentRepository.getCachedClass() never gets called, so the UCL doesn't have access to the repository'scached classes.

Within the scope of the call to getPackageClassLoaders() at Step 3, the child repository also callsgetPackageClassLoaders() on its parent, and also includes into the returned class loader set a UCL (constructed on the spot and associated to the child repository) that has among its ancestors an instance ofNoAnnotationURLClassLoader, which ultimately can reach the system class loader. Why is that? Remember that the UCL's parent, HierarchicalLoaderRepository3$NoParentClassLoader, overrides loadClass() to always throw a ClassNotFoundException, thus forcing the UCL to only load from its URLs. If the UCL relies only on its class loader parent to load bootstrap classes, it will throw ClassNotFoundException and fail when your application wants to load "java.lang.String", for example. The NoAnnotationURLClassLoader-delegating UCL instance included in the return set provides a way load bootstrap library classes.

Always the HierarchialLoaderRepository's class loaders take precedence over the parent's (their "order" is lower). For the case depicted above, UCL1 is the preferred class loader.

按原文描述的意思大致就是在应用的UCL加载类的时候不会去调用父Repository的getCachedClass()方法,在执行loadClassImpl()方法是会调用父getCachedClass()的getPackageClassLoaders()方法。但是即便返回的ucl集合包含父Repository的ucl HierarchialLoaderRepository3也会保证自己包含的UCL的优先级高于父类的UCL。
具体如何修改配置实现如上所说的隔离效果请看http://community.jboss.org/wiki/ClassLoadingConfiguration的说明

但是在我实际操作的过程中并没有上面的文档说的这么简单。在做了如上的操作以后应用启动失败，有个ejb部署失败（准确来说是一个MDB）报错如下：

2010-12-11 23:39:23,513 ERROR [org.jboss.deployment.MainDeployer] Could not create deployment: file:/home/sean/output/logs/jboss_stout/tmp/deploy/tmp869229660270410713xxxx-ear.ear-contents/lib/xxxx.components.financestat-1.0.jar
org.jboss.deployment.DeploymentException: Verification of Enterprise Beans failed, see above for error messages.
        at org.jboss.ejb.EJBDeployer.create(EJBDeployer.java:610)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
        at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
        at org.jboss.mx.interceptor.AbstractInterceptor.invoke(AbstractInterceptor.java:133)

对于这一块报错的原因，由于日志提示的不是很明确，当时时间也比较紧就没有再研究下。如果各位了解此问题的话望不吝赐教。

下面我介绍一下我最后使用的解决办法:
因为前面也介绍了导致问题的原因是由于：

org.jboss.mx.loading.UnifiedClassLoader3@1fe500a{ url=file:/home/sean/web-deploy/jboss_server/default/deploy/jbossws.sar/ ,addedOrder=12}的优先级高于ucl: org.jboss.mx.loading.UnifiedClassLoader3@16274ea{ url=file:/home/sean/output/logs/jboss_stout/tmp/deploy/tmp7754042895360577419xxxx-ear.ear ,addedOrder=45}的优先级。那么如果将jbossws.sar的Repository和jboss的Repository进行隔离的话这样在jboss的Repository里就不存在jettison.jar了。因为我们的应用使用不到jboss webservice功能，所以这个改动不会影响到我们的程序。
具体配置方法是在$JBOSS_SERVER_HOME/deploy/jbossws.sar/META-INF/jboss-service.xml里添加如下配置项

<loader-repository>
          com.example:archive=nouse
   </loader-repository>

现在再来看一下日志，已经是我们想要的效果了。

[116920,RepositoryClassLoader,DefaultQuartzScheduler_Worker-8] loadClass org.jboss.mx.loading.UnifiedClassLoader3@abe02e{ url=file:/home/sean/output/logs/jboss_stout/tmp/deploy/t
mp429376920320337088xxxx-ear.ear ,addedOrder=45} name=org.codehaus.jettison.AbstractXMLStreamWriter class=class org.codehaus.jettison.AbstractXMLStreamWriter cl=org.jboss.mx.lo
ading.UnifiedClassLoader3@abe02e{ url=file:/home/sean/output/logs/jboss_stout/tmp/deploy/tmp429376920320337088xxxx-ear.ear ,addedOrder=45}

以上内容是我通过查看jboss文档和jboss、tomcat相关代码后对jboss、tomcat的类加载机制的不完全分析。其中可能有些理解错误的地方，欢迎大家斧正。