准备:
当你的所有数据非常多,一台服务器无法承担,这时Sharding是一种常见的扩展解决策略,即将这些索引数据分散到不同的服务器上,而查询时Solr具备将单一的查询请求分发到多个Solr shards上,并聚集结果到一个单一的result里返回调用者。
建立分布式索引需遵循以下条件:
索引的文档必须有一个唯一键; 当拥有相同的id的文档被查询到时,solr将选择第一个文档。其他的结果将被丢弃;在处理的过程中,也即是 stage过程中,索引是有可能发生变化的。这样在索引更新的时刻,可能会发现搜索的结果跟索引不匹配的现象。
实例描述:
操作系统:WindowsXP
Web容器:Tomcat
同时启动三个Tomcat模拟三台服务器,分配端口号分别为:8080,8888,8983
Solr配置:
注意:分布式索引不需要额外配置,已经被绑定在原有的Request Handlers上,
开始:
Step one: 向8888端口号服务器添加索引数据,原始数据见附件索引中kobe.xml
kobe.xml明细:
<add>
<doc>
<field name="id">107</field>
<field name="name">LBJ is a basketball player</field>
</doc>
<doc>
<field name="id">108</field>
<field name="name">taylor is a singer</field>
</doc>
<doc>
<field name="id">109</field>
<field name="name">kaku is a actor</field>
</doc>
<doc>
<field name="id">110</field>
<field name="name">tracy is a basketball player</field>
</doc>
<doc>
<field name="id">111</field>
<field name="name">Yao is a basketball player</field>
</doc>
<doc>
<field name="id">112</field>
<field name="name">me is a actor</field>
</doc>
</add>
Step two: 向8983端口号服务器添加索引数据,原始数据见附件索引中jane.xml
jane.xml代码明细:
<add>
<doc>
<field name="id">107</field>
<field name="name">LBJ is a basketball player</field>
</doc>
<doc>
<field name="id">108</field>
<field name="name">taylor is a singer</field>
</doc>
<doc>
<field name="id">109</field>
<field name="name">kaku is a actor</field>
</doc>
<doc>
<field name="id">110</field>
<field name="name">tracy is a basketball player</field>
</doc>
<doc>
<field name="id">111</field>
<field name="name">Yao is a basketball player</field>
</doc>
<doc>
<field name="id">112</field>
<field name="name">me is a actor</field>
</doc>
</add>
Step three: 测试,用浏览器作为客户端测试:
测试1: http://localhost:8888/solr/select/?q=id:102 测试结果如下:
<?xml version="1.0" encoding="UTF-8" ?>
- <response>
- <lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">0</int>
- <lst name="params">
<str name="q">id:102</str>
</lst>
</lst>
- <result name="response" numFound="1" start="0">
- <doc>
<str name="id">102</str>
<str name="name">kobe bryant is a basketball player</str>
</doc>
</result>
</response>
测试2: http://localhost:8983/solr/select/?q=id:112测试结果
<?xml version="1.0" encoding="UTF-8" ?>
- <response>
- <lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">15</int>
- <lst name="params">
<str name="q">id:112</str>
</lst>
</lst>
- <result name="response" numFound="1" start="0">
- <doc>
<str name="id">112</str>
<str name="name">me is a actor</str>
</doc>
</result>
</response>
测试3: http://localhost:8080/solr/select/?shards=localhost:8888/solr,localhost:8983/solr&q=singer测试结果:
<?xml version="1.0" encoding="UTF-8" ?>
- <response>
- <lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">16</int>
- <lst name="params">
<str name="shards">localhost:8888/solr,localhost:8983/solr</str>
<str name="q">singer</str>
</lst>
</lst>
- <result name="response" numFound="4" start="0">
- <doc>
<str name="id">108</str>
<str name="name">taylor is a singer</str>
</doc>
- <doc>
<str name="id">103</str>
<str name="name">jay chou is a singer</str>
</doc>
- <doc>
<str name="id">104</str>
<str name="name">jane zhang is a singer</str>
</doc>
- <doc>
<str name="id">105</str>
<str name="name">coco lee is a singer</str>
</doc>
</result>
</response>
测试4: http://localhost:8080/solr/select/?shards=localhost:8888/solr,localhost:8983/solr&q=player 测试结果
<?xml version="1.0" encoding="UTF-8" ?>
- <response>
- <lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">16</int>
- <lst name="params">
<str name="shards">localhost:8888/solr,localhost:8983/solr</str>
<str name="q">player</str>
</lst>
</lst>
- <result name="response" numFound="4" start="0">
- <doc>
<str name="id">102</str>
<str name="name">kobe bryant is a basketball player</str>
</doc>
- <doc>
<str name="id">107</str>
<str name="name">LBJ is a basketball player</str>
</doc>
- <doc>
<str name="id">110</str>
<str name="name">tracy is a basketball player</str>
</doc>
- <doc>
<str name="id">111</str>
<str name="name">Yao is a basketball player</str>
</doc>
</result>
</response>
Step four :分析
测试1和测试2返回的结果都是从本机上索引文件获得的,而测试3和测试4是分布式查询,它所对应主机为8080,该服务武器上没有数据,这就是分布式查询(向8888和8983),将所有结果从不同Tomcat上聚合后返回
也可以根据前面学习笔记(三)中提供的SolrjClient来测试分布式索引。下面在三的基础上,开启三个Tomcat,模拟三台服务器,端口布置如上所示,开启三台服务器后,利用SolrjClient提供的接口分别向三台服务器提供索引数据如下:
向8080服务器:
List<SolrInputDocument> docs = new ArrayList<SolrInputDocument>();
for (int i = 0; i < 10; i++) {
SolrInputDocument doc = new SolrInputDocument();
doc.addField("id", i);
doc.addField("severity",i);
doc.addField("sLocation","Beijing_" + i);
doc.addField("msg", "kobe bryant" + i);
doc.addField("name", "kylin soong" + i);
docs.add(doc);
}
client.addDocs(docs);
向8888服务器添加:
List<SolrInputDocument> docs = new ArrayList<SolrInputDocument>();
for (int i = 10; i < 20; i++) {
SolrInputDocument doc = new SolrInputDocument();
doc.addField("id", i);
doc.addField("severity",i);
doc.addField("sLocation","Beijing_" + i);
doc.addField("msg", "kobe bryant" + i);
doc.addField("name", "kylin soong" + i);
docs.add(doc);
}
client.addDocs(docs);
向8983服务器添加:
List<SolrInputDocument> docs = new ArrayList<SolrInputDocument>();
for (int i = 20; i < 30; i++) {
SolrInputDocument doc = new SolrInputDocument();
doc.addField("id", i);
doc.addField("severity",i);
doc.addField("sLocation","Beijing_" + i);
doc.addField("msg", "kobe bryant" + i);
doc.addField("name", "kylin soong" + i);
docs.add(doc);
}
client.addDocs(docs);
然后在利用SolrjClient提供接口进行查询验证(在8080服务器上查询):
String[] shards = new String[]{"localhost:8080/solr","localhost:8888/solr","localhost:8983/solr"};
String[] fields = new String[]{"id", "name", "msg"};
List<SolrDocument> lists = client.query("kobe", 30, fields, shards);
System.out.println(lists.size());
show(lists);
结果如下:
30
id=0 msg=kobe bryant0 name=kylin soong0
id=1 msg=kobe bryant1 name=kylin soong1
id=2 msg=kobe bryant2 name=kylin soong2
id=3 msg=kobe bryant3 name=kylin soong3
id=4 msg=kobe bryant4 name=kylin soong4
id=5 msg=kobe bryant5 name=kylin soong5
id=6 msg=kobe bryant6 name=kylin soong6
id=7 msg=kobe bryant7 name=kylin soong7
id=8 msg=kobe bryant8 name=kylin soong8
id=9 msg=kobe bryant9 name=kylin soong9
id=10 msg=kobe bryant10 name=kylin soong10
id=11 msg=kobe bryant11 name=kylin soong11
id=12 msg=kobe bryant12 name=kylin soong12
id=13 msg=kobe bryant13 name=kylin soong13
id=14 msg=kobe bryant14 name=kylin soong14
id=15 msg=kobe bryant15 name=kylin soong15
id=16 msg=kobe bryant16 name=kylin soong16
id=17 msg=kobe bryant17 name=kylin soong17
id=18 msg=kobe bryant18 name=kylin soong18
id=19 msg=kobe bryant19 name=kylin soong19
id=20 msg=kobe bryant20 name=kylin soong20
id=21 msg=kobe bryant21 name=kylin soong21
id=22 msg=kobe bryant22 name=kylin soong22
id=23 msg=kobe bryant23 name=kylin soong23
id=24 msg=kobe bryant24 name=kylin soong24
id=25 msg=kobe bryant25 name=kylin soong25
id=26 msg=kobe bryant26 name=kylin soong26
id=27 msg=kobe bryant27 name=kylin soong27
id=28 msg=kobe bryant28 name=kylin soong28
id=29 msg=kobe bryant29 name=kylin soong29
如上结果把符合查询字符串“Kobe”的所有字符串都返回,
PS:上述相关代码见学习笔记(三)附件
附件上XML文件与本文示例有些许差别……
结束
分享到:
相关推荐
`xufuli.txt`可能是一个用户自定义的文档,可能是Solr学习笔记、配置示例或者使用心得。不过,由于具体内容未知,这里无法详细展开。 总之,Solr 4.10.3是一个强大且灵活的搜索引擎,适用于各种规模的企业级应用。...
doc.addField("title", "Solr学习笔记"); client.add(doc); client.commit(); ``` 对于查询操作,你可以使用`SolrQuery`类来构造查询条件,然后调用`query()`方法执行查询: ```java SolrQuery query = new ...
**Lucene学习笔记** Lucene是一个高性能、全文检索库,由Apache软件基金会开发并维护,是Java编程语言中广泛使用的搜索引擎库。它提供了一个简单但功能强大的API,用于索引和搜索文本数据,使得开发者可以轻松地在...
压缩包中的“Nutch.pdf”可能是Nutch的官方文档、教程或者是博主整理的学习笔记。这份PDF文档通常会详细讲解Nutch的架构、工作流程、配置方法,以及一些示例和最佳实践。学习者可以通过阅读这份文档,系统地了解...