Solr4.0 tomcat集群和elasticSearch一些事

  最近业余时间接触es不久，碰到了一些关于批量添加数据的问题，在本地机器上配置了三个节点的集群，一共有5.2w的元数据，批量更新的时候尼玛总是出现IO异常问题，数据少还中，比如500，1000。已经做了批量更新的间隔，如一次1000,500,200都试过，还是IO。如果说是本机配置的不够，2G的足够了吧，通过一个侧面的例子也能反应出来，同样在本机做了一个numshards=3的solr集群，更新25.8w的数据都没有问题。

  下面贴出创建es索引，更新数据的代码，有懂的朋友帮解答一下，ps: es的资料太少了，光看例子都够费劲的，尼玛java链式方法老长老长，显得牛逼吗。

  

public void createIndex() throws IOException {
		XContentBuilder content = XContentFactory.jsonBuilder()
												.startObject()
												.startObject("vType")
												.startObject("properties")
												.startObject("title").field("type", "string").field("indexAnalyzer", "ik").field("searchAnalyzer", "ik").endObject()
												.startObject("author").field("type", "string").field("indexAnalyzer","ik").field("searchAnalyzer", "ik").endObject()
												.startObject("keyword").field("type", "string").field("indexAnalyzer","ik").field("searchAnalyzer", "ik").endObject()
												.startObject("fenlei").field("type", "string").endObject()
												.startObject("other").field("type", "string").endObject()
												.startObject("subname").field("type", "string").endObject()
												.startObject("subtitles").field("type", "string").endObject()
												.startObject("summary").field("type", "string").field("indexAnalyzer","ik").field("searchAnalyzer", "ik").endObject()
												.startObject("videolink").field("type", "string").endObject()
												.endObject()
												.endObject()
												.endObject();
		client.admin().indices().prepareCreate("video").addMapping("vType", content)
						          .setSettings(ImmutableSettings.settingsBuilder().put("number_of_shards", 8)).execute().actionGet();
	}

 

     更新数据的代码：

    

public void bulkData(){
		BulkRequestBuilder blukRequest = client.prepareBulk();
		try {
		IndexReader reader = IndexReader.open(FSDirectory.open(new File(store_path)));
		int maxDocs = reader.maxDoc();
		List<Document> list  = new ArrayList<Document>();
		int count = 0;
		for (int i = 0; i < maxDocs; i++) {
				list.add(reader.document(i));
				if(list.size()%100==0){
					for(int j =0;j<list.size();j++){
						Document doc = list.get(j);
						blukRequest.add(client.prepareIndex("video", "vType", String.valueOf((j+count)))
								   .setSource(XContentFactory.jsonBuilder()
										                     .startObject()
										                     .field("title",doc.get("title"))
										                     .field("author",doc.get("author"))
										                     .field("keyword",doc.get("keyword"))
										                     .field("fenlei",doc.get("fenlei"))
										                     .field("other",doc.get("other"))
										                     .field("subname",doc.get("subname"))
										                     .field("subtitles",doc.get("subtitles"))
										                     .field("summary",doc.get("summary"))
										                     .field("videolink", doc.get("videolink"))
										                     .endObject()
										                     ));
					}
					count+=list.size();
					BulkResponse bulkResponse = blukRequest.execute().actionGet();
					client.admin().indices().prepareRefresh().execute().actionGet();
					if(!bulkResponse.hasFailures())
						System.out.println("------success!");
					else
						System.out.println("------failure!");
					list.clear();
				}
		 }
		if(list.size()>0){
			for(int i =0;i<list.size();i++){
				Document doc = list.get(i);
				blukRequest.add(client.prepareIndex("video", "vType", String.valueOf((i+count)))
						   .setSource(XContentFactory.jsonBuilder()
								                     .startObject()
								                     .field("title",doc.get("title"))
								                     .field("author",doc.get("author"))
								                     .field("keyword",doc.get("keyword"))
								                     .field("fenlei",doc.get("fenlei"))
								                     .field("other",doc.get("other"))
								                     .field("subname",doc.get("subname"))
								                     .field("subtitles",doc.get("subtitles"))
								                     .field("summary",doc.get("summary"))
								                     .field("videolink", doc.get("videolink"))
								                     .endObject()
								                     ));
				count+=list.size();
				BulkResponse bulkResponse = blukRequest.execute().actionGet();
//				client.admin().indices().prepareRefresh().execute().actionGet();
				if(!bulkResponse.hasFailures())
					System.out.println("------success!");
				else
					System.out.println("------failure!");
				list.clear();
			}
		}
		
		} catch (CorruptIndexException e) {
		} catch (IOException e) {
			e.printStackTrace();
		}
	}

   还是静坐等待高手的指点ing......

   接下来介绍solr4.0Alpha,喜欢solr的朋友可以学习了哦...

   限于篇幅solr4.0+tomcat的安装，查看我的百度空间http://hi.baidu.com/620734263/item/372ea2c7955fbc7088ad9eaa