`
kylinsoong
  • 浏览: 239676 次
  • 性别: Icon_minigender_1
  • 来自: 北京
社区版块
存档分类
最新评论

Lucene/Solr Dev 3 Solr Cache and Load Balance

阅读更多

Warmming up:

      Analysing the solr src class, which packaged in org.apache.solr.search. There are two implementations of cache available for Solr, LRUCache, based on a synchronized LinkedHashMap, and FastLRUCache, based on a ConcurrentHashMap. FastLRUCache has faster gets and slower puts in single threaded operation and thus is generally faster than LRUCache when the hit ratio of the cache is high (> 75%). And then i will delve into the whloe solr's cache, let's start from FastLRUCache FIFO strategy.

FastLRUCache FIFO strategy:

     FIFO means first come first in, which is a very common strategy in software design, i will give a experiment directly to account for FIFO strategy.

     In order to make the experiment result more persuasively, we should add a row code in Solr's org.apache.solr.common.util.ConcurrentLRUCache<K,V>, we add the below code

 

	  //add by kylin
	  System.out.println(key +  " " + map.keySet());

 

to ConcurrentLRUCache's put(K key, V val), yes, it's just a output sentence, use to output the cache's size before a put has coming.

and then we run the below code;

public void testFastLRUCacheFIFO() {
		FastLRUCache cache = new FastLRUCache();
		Map map = new HashMap();
		map.put("size", "3");
		CacheRegenerator regenerator = new CacheRegenerator() {
			public boolean regenerateItem(SolrIndexSearcher newSearcher,
					SolrCache newCache, SolrCache oldCache, Object oldKey,
					Object oldVal) throws IOException {
				newCache.put(oldKey, oldVal);
				return true;
			}};
		
		Object obj = cache.init(map, null, regenerator);
		cache.setState(SolrCache.State.LIVE);
		
		for (int i = 1; i < 10; i++) {
			cache.put(i , "" + i);
		}
	}

 map.put("size", "3"), this is very imporant, is the key factor of  FastLRUCache FIFO, implement the code, and the result is:

 

1 [1]
2 [2, 1]
3 [2, 1, 3]
4 [3, 4]
5 [5, 3, 4]
6 [6, 5]
7 [6, 5, 7]
8 [7, 8]
9 [7, 8, 9]

 Through the result data, we can know that the latest record was saved, but this not totally FIFO, because the Out number is count by the size you have set: Out numer = size * 10%, but the minimal Out size is 2. so we can count the above Out number(3 * 10% = 0, 0 < 2, so Out num is 2).

 

change the above code, we look at the warming mechenism.

public void test() throws IOException {
		FastLRUCache cache = new FastLRUCache();
		Map map = new HashMap();
		map.put("size", "100");
		map.put("initialSize", "10");
		map.put("autowarmCount", "30");
		
		CacheRegenerator regenerator = new CacheRegenerator() {
			public boolean regenerateItem(SolrIndexSearcher newSearcher,
					SolrCache newCache, SolrCache oldCache, Object oldKey,
					Object oldVal) throws IOException {
				newCache.put(oldKey, oldVal);
				return true;
			}};
		
		Object obj = cache.init(map, null, regenerator);
		cache.setState(SolrCache.State.LIVE);
		
		for (int i = 1; i < 102; i++) {
			cache.put(i , "" + i);
		}
	
		System.out.println(cache.get(10));
		System.out.println(cache.get(11));
		
		FastLRUCache cacheNew = new FastLRUCache();
		cacheNew.init(map, obj, regenerator);
		cacheNew.warm(null, cache);
		cacheNew.setState(SolrCache.State.LIVE);
		cache.close();
		cacheNew.put(103, "103");
		System.out.println(cacheNew.get(72));
		System.out.println(cacheNew.get(73));
		
	}

 and also i give the implement result:

null
11
null
73

  Anaysing the result:

    The size = 100, so 100 * 10% = 10, OutNumber = 10, when the 101th recod has added, the first came 10 recod were remove, so the null printed, and then 11 was printed.

    The autowarmCount = 30, so when a new SolrSearcher coming, it will be bould with new Searcher, 102-30=72, So when you get 72 in new Searcher, the null is printed,and if you get 73, the responding value is printed.

 

 

 

 

 

 

 

 

 

 

 

 

分享到:
评论

相关推荐

    为Lucene/Solr 4.7重新打包的IKAnalyzer 2012 FF

    当前的IKAnalyzer官方版在用于Solr4以上高版本时,由于没有TokenizerFactory而造成诸多不便,于是有了为Lucene/Solr 4.7重新打包的IKAnalyzer 2012 FF

    mmseg4j-solr总共4个文件

    mmseg4j-solr-2.0.0.jar 要求 lucene/solr &gt;= 4.3.0。在 lucene/solr [4.3.0, 4.7.1] 测试过兼容可用。 mmseg4j-solr-2.1.0.jar 要求 lucene/solr 4.8.x mmseg4j-solr-2.2.0.jar 要求 lucene/solr [4.9, 4.10.x] ...

    lucene&solr原理分析

    lucene&solr原理分析,lucene搜索引擎和solr搜索服务器原理分析。

    apache lucene solr 官网历史版本 免费下载地址

    http://archive.apache.org/dist/lucene/java/ 这个是lucene的历史版本 http://archive.apache.org/dist/lucene/solr/ 这个是solr的历史版本

    lucene,solr的使用

    ### Lucene与Solr的使用详解 #### 一、Lucene概述 Lucene是一款高性能、全功能的文本搜索引擎库,由Java语言编写而成。它能够为应用系统提供强大的全文检索能力,是当前最为流行的开源搜索库之一。由于其高度可...

    Apache+Solr+Reference+Guide 2018.pdf

    It does not assume that you are a Java programmer, although knowledge of Java is helpful when working directly with Lucene or when developing custom extensions to a Lucene/Solr installation.

    解决solr启动404问题

    Solr是Apache Lucene项目的一个子项目,是一个高性能、基于Java的企业级全文搜索引擎服务器。当你在尝试启动Solr时遇到404错误,这通常意味着Solr服务没有正确地启动或者配置文件设置不正确。404错误表示“未找到”...

    Lucene&solr.zip

    3. 创建索引:使用Lucene API或Solr的索引API,将数据转化为索引结构。 4. 查询:通过Solr的HTTP接口发送查询请求,获取搜索结果。 5. 维护:定期更新索引,监控系统性能,优化查询效率。 总的来说,Lucene和Solr是...

    LoremIpsumSearch:包含与 lucene 和 solr 一起使用的搜索算法

    LoremIpsum搜索 包含与 lucene 和 solr 一起使用的搜索算法... export CLASSPATH="&lt;lucene&gt;/lucene/replicator/lib/*:&lt;nutch&gt;/build/*:&lt;nutch&gt;/build/lib/*:&lt;lucene&gt;/solr/dist/*:&lt;lucene&gt;/solr/ dist/solrj-lib/*:*:.

    在tomcat环境下搭建solr3.5和mmseg4j搜索引擎

    - 访问官方下载页面:[http://www.apache.org/dyn/closer.cgi/lucene/solr/](http://www.apache.org/dyn/closer.cgi/lucene/solr/) - 选择版本3.5并将其解压到D盘,例如路径为`D:/solr/apache-solr-3.5.0` 2. **...

    lucene-solr-analysis-turkish:Apache LuceneSolr的土耳其语分析组件

    适用于Apache Lucene / Solr的土耳其语分析组件 在土耳其,开源软件的使用正日益增长。 Apache Lucene / Solr(和其他 )邮件列表上的土耳其用户正在增加。 该项目利用公共可用的土耳其语NLP工具从中创建。 我创建...

    lucene-solr源码,编译成的idea项目源码

    本人用ant idea命令花了214分钟,35秒编译的lucene-solr源码,可以用idea打开,把项目放在D:\space\study\java\lucene-solr路径下,再用idea打开就行了

    solr -8.11.1.zip 文件

    solr -8.11.1.zip 文件

    IKAnalyzer 适用 lucene和solr 5.4.0版本

    3. **配置Lucene或Solr**:将生成的jar包添加到Lucene或Solr的lib目录下,然后在相应配置文件中指定IKAnalyzer为Analyzer。 4. **测试与应用**:编写测试代码验证IKAnalyzer的分词效果,确认无误后即可在实际项目中...

Global site tag (gtag.js) - Google Analytics