SolrCore查询过程源码分析

一.环境启动代码

SolrResourceLoader.SetSolrHome("/duitang/dist/app/branches/mdrill/trunk/adhoc-core/solr");
SolrResourceLoader.SetSchemaHome("/home/taobao/solr/schema");

ShardPartion.base = "/group/tbdp-etao-adhoc/p4padhoc/tabletest"; //必须的，如果不配置会有异常,SolrCore中会使用ShardPartion.getHdfsRealtimePath
ShardPartion.taskIndex = 1;
ShardPartion.index = IndexUtils.getHdfsForder(1);

String hdfsconf = "/duitang/dist/sys/hadoop-1.2.1/conf/"; //必须的，如果不配置会有异常,ReadOnlyDirectory.getConf()
HadoopUtil.setHdfsConfDir(hdfsconf);

JettySolrRunner jetty = new JettySolrRunner("/solr", 1210);
jetty.start();
while (true)
{
	Thread.sleep(1000);
}

 

二.启动之后通过URL测试

http://localhost:1210/solr/select?q=*:*&indexpartion=20111014

 

三.查询入口SolrDispatchFilter

入口在SolrDispatchFilter中，通过拦截请求中的select来加载对应的requestHandler.代码如下:

 

init():
	CoreContainer.Initializer init = new CoreContainer.Initializer();
	CoreContainer cores = init.initialize();
	core = cores.getCore("");

doFilter():
	final SolrConfig config = core.getSolrConfig();
	SolrRequestParsers parser = new SolrRequestParsers(config);
	SolrQueryRequest solrReq = parser.parse( core, "select", httpServletRequest)
	SolrRequestHandler handler = core.getRequestHandler(null)

	SolrQueryResponse solrRsp = new SolrQueryResponse();
	solrReq.getContext().put( "webapp", req.getContextPath() );
	solrReq.getCore().execute( handler, solrReq, solrRsp );

 

 

四.核心方法SolrCore.execute()

入口SolrCore.execute(),前面都是web请求忽略掉。

SolrCore.execute(SolrRequestHandler handler, SolrQueryRequest req, SolrQueryResponse rsp)执行过程：

SolrCore.execute(SolrRequestHandler handler, SolrQueryRequest req, SolrQueryResponse rsp)
-->handler.handleRequest(req, rsp);
-->setResponseHeaderValues(handler, req, rsp);

 

 

SolrCore实际调用的是SearchHandler.handleRequest(),SearchHandler继承于RequestHandlerBase，覆盖了handleRequestBody(),调用关系如下：

-->RequestHandlerBase.handleRequest(SolrQueryRequest req, SolrQueryResponse rsp)
	-->SearchHandler.handleRequestBody(SolrQueryRequest req, SolrQueryResponse rsp)

 

SearchHandler.handleRequestBody的实现比较复杂，如果shards不为null，会尝试请求各shards然后对结果合并。这里先看shards为null的情况：

-->QueryComponent.process(ResponseBuilder rb)
	-->SolrIndexSearcher searcher = rb.req.getSearcher();
	-->searcher.search(result,cmd);
-->FacetComponent.process(ResponseBuilder rb)

 req是SolrQueryRequest类型，实际是SolrQueryRequestBase实例，看看其getSearcher()实现方法。

1.SolrQueryRequestBase.getSearcher()
-->String partion=this.getParams().get(CommonParams.PARTION,"default");
-->SolrCore.getSearcher(partion,false,false,higohb);

2.SolrCore.getSearcher(String partion, boolean mdirs, boolean isclearCache, boolean ishb
-->String partionKey=this.corename+"@"+partion+"@"+SolrResourceLoader.getCacheFlushKey(p);
-->SolrCore.getSearcherByPath(p,partionKey, f.getAbsolutePath(), mdirs, isclearCache, ishb)

3.SolrCore.getSearcherByPath(PartionKey p, String partionKey, String path):

if partion.name == "default":
	//初始化IndexReader
	RAMDirectory rd=new RAMDirectory();
	rd.setCore(this,p);
	IndexWriter writer=new IndexWriter(rd, null,new KeepOnlyLastCommitDeletionPolicy(), MaxFieldLength.UNLIMITED);
	writer.setMergeFactor(10);
	writer.setUseCompoundFile(false);
	writer.close();	
	reader=IndexReader.open(rd);

	//初始化SolrIndexSearcher
	SolrIndexSearcher newSearcher = new SolrIndexSearcher(this,	schema, "partion_" + partionKey, reader, true);
	newSearcher.setPartionCacheKey(p);
	newSearcher.setFieldcacheDir(fieldcacheDir);
	newSearcher.setPartionKey(partionKey);
	rtn = newHolderPartion(newSearcher);

	searchCache.put(partionKey, rtn);

else:
	//初始化IndexReader
	MdrillDirectory rtdir = new ReadOnlyDirectory(f, HadoopUtil.hadoopConfDir, ShardPartion.getHdfsRealtimePath(p.tablename, p.partion));
	rtdir.getForSearch()
	fs.mkdirs(new Path(hdfsPath).getParent());	//hdfs_path-->	/group/.../20110111
	path.mkdirs();								//local_path-->	/duitang/.../20110111
	File links=new File(path,"indexLinks");
	while f=links.readline():
		FSDirectory d=LinkFSDirectory.open(f);	
		DirectoryInfo info=new DirectoryInfo();
		info.d=d;
		info.tp=DirectoryInfo.DirTpe.file;
		diskDirector.put(f, info);
	
	IndexReader[] r = new IndexReader[diskDirector.size()];
	for	directory in diskDirector.values:
		r[i] = IndexReader.open(directory)
	
	if r.length==1:
		reader = r[0]
	else:
		reader = new MultiReader(r, true);

	//初始化SolrIndexSearcher(同上)

 可以看到如果indexpartion没有传递默认是default,开辟空的内存，这时没有索引被读取。当有indexpartion时会读取indexpartion目录下的indexLinks。

比如/duitang/dist/app/branches/mdrill/trunk/adhoc-core/solr/data/20110111/indexLinks。indexLinks会记录索引存放的路径，然后分别加载，最后通过MultiReader实现。

 

索引测试路径：

local_path:/duitang/dist/app/branches/mdrill/trunk/adhoc-core/solr/data/20110111

hdfs_path:/group/tbdp-etao-adhoc/p4padhoc/tabletest/index/20110111/part-00001

 

 

最后补充solr容器的关闭,下次分析

"Shutdown" prio=10 tid=0x0000000040e97000 nid=0x7b1d at breakpoint[0x00007f7af4772000]
   java.lang.Thread.State: RUNNABLE
	at org.apache.solr.core.SolrCore.close(SolrCore.java:473)
	at org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:202)
	- locked <0x00000000e4288b70> (a java.util.LinkedHashMap)
	at org.apache.solr.servlet.SolrDispatchFilter.destroy(SolrDispatchFilter.java:104)
	at org.mortbay.jetty.servlet.FilterHolder.destroyInstance(FilterHolder.java:127)
	at org.mortbay.jetty.servlet.FilterHolder.doStop(FilterHolder.java:107)
	at org.mortbay.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:76)
	- locked <0x00000000e4280868> (a java.lang.Object)
	at org.mortbay.jetty.servlet.ServletHandler.doStop(ServletHandler.java:176)
	- locked <0x00000000e4280768> (a org.mortbay.jetty.servlet.ServletHandler)
	at org.mortbay.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:76)
	- locked <0x00000000e42807d8> (a java.lang.Object)
	at org.mortbay.jetty.handler.HandlerWrapper.doStop(HandlerWrapper.java:142)
	at org.mortbay.jetty.servlet.SessionHandler.doStop(SessionHandler.java:125)
	at org.mortbay.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:76)
	- locked <0x00000000e4280758> (a java.lang.Object)
	at org.mortbay.jetty.handler.HandlerWrapper.doStop(HandlerWrapper.java:142)
	at org.mortbay.jetty.handler.ContextHandler.doStop(ContextHandler.java:592)
	at org.mortbay.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:76)
	- locked <0x00000000e4280708> (a java.lang.Object)
	at org.mortbay.jetty.handler.HandlerWrapper.doStop(HandlerWrapper.java:142)
	at org.mortbay.jetty.Server.doStop(Server.java:283)
	at org.mortbay.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:76)
	- locked <0x00000000e4280658> (a java.lang.Object)
	at org.mortbay.jetty.Server$ShutdownHookThread.run(Server.java:561)

  

补充SolrCore初始化的stacktrace，下次分析

"2010005445@qtp-385415131-0" daemon prio=10 tid=0x0000000041df3800 nid=0x440c at breakpoint[0x00007f2686060000]
   java.lang.Thread.State: RUNNABLE
	at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:278)
	at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:85)
	at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:808)
	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:91)
	at org.apache.lucene.index.IndexReader.open(IndexReader.java:509)
	at org.apache.lucene.index.IndexReader.open(IndexReader.java:369)
	at org.apache.lucene.search.IndexSearcher.<init>(IndexSearcher.java:110)
	at org.apache.lucene.search.spell.SpellChecker.createSearcher(SpellChecker.java:752)
	at org.apache.lucene.search.spell.SpellChecker.swapSearcher(SpellChecker.java:729)
	at org.apache.lucene.search.spell.SpellChecker.setSpellIndex(SpellChecker.java:174)
	- locked <0x00000000e25f3818> (a java.lang.Object)
	at org.apache.lucene.search.spell.SpellChecker.<init>(SpellChecker.java:149)
	at org.apache.solr.spelling.AbstractLuceneSpellChecker.init(AbstractLuceneSpellChecker.java:127)
	at org.apache.solr.spelling.IndexBasedSpellChecker.init(IndexBasedSpellChecker.java:55)
	at org.apache.solr.handler.component.SpellCheckComponent.inform(SpellCheckComponent.java:598)
	at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:523)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:385)
	at org.apache.solr.core.CoreContainer.createCore(CoreContainer.java:180)
	at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:151)
	- locked <0x00000000e2e23bf0> (a java.util.LinkedHashMap)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:148)