`

hadoop+lucene+web 综合小demo

hadoopluceneweb设计应用 
阅读更多

很长一段时间没有关注hadoop，突然间有兴致，于是动手又研究一下

准备的基础如下:

下载hadoop-1.0.4,tomcat,lucene由于关于hadoop的基础资料也挺多的，所以这里只写一段关于hadoop+lucene+web小综合的demo设计思路， 希望对初学入门者有所启示，如下图:

hadoop+lucene+servlet+jsp整合

 

本例子会引用最初hadoop提供的wordcount例子,具体的代码可以通过网上得到,然后运行.

workcount输入文件名称inputfile.txt 包含内容：

Hello World Bye World goole

workcount输出的文件名称part-00000 ，包含内容：

Bye 1
Hello 1
World 2
goole 1

假设:在生成part-00000 文件之后,马上调用lucene创建索引文件

创建索引文件的java代码如下：

 // @author minn

IndexWriter writer = new IndexWriter(PRE_PATH+ "index",new SimpleAnalyzer(),true);
			FileReader read = new FileReader(PRE_PATH+"part-00000");
			BufferedReader br = new BufferedReader(read);
			String row;
			Document document =null;
		    while((row = br.readLine())!=null){
				String tmp[]=row.split("\\s+");
				document=new Document();
		    	document.add(new Field("name",tmp[0],Field.Store.YES,Field.Index.TOKENIZED));
		    	document.add(new Field("count",tmp[1],Field.Store.YES,Field.Index.TOKENIZED));
				writer.addDocument(document);
			   }
			writer.optimize();
			writer.close();

 

查找java代码：

/**
 * 
 * @author minn
 *
 */
public static String searchWord(String word) throws Exception {
		Analyzer analyzer = new StandardAnalyzer();
		String result = "";
		IndexSearcher indexSearcher = null;

		indexSearcher = new IndexSearcher(PRE_PATH + "index");

		QueryParser queryParser = new QueryParser("name", analyzer);
		Query query = null;

		query = queryParser.parse(word);

		if (null != query && null != indexSearcher) {
			Hits hits = indexSearcher.search(query);

			result = hits.doc(0).get("count");

		}
		return result;
	}

 

servlet调用lucene查找：

package servlet;

import java.io.IOException;
import java.io.PrintWriter;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.ArrayList;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import lucene.SearchHadoopFile;



/**
 * Servlet implementation class HadoopServlet
 * @author minn
 */
public class HadoopServlet extends HttpServlet {
	private static final long serialVersionUID = 1L;
       

    /**
     * @see HttpServlet#HttpServlet()
     */
    public HadoopServlet() {
        super();
        // TODO Auto-generated constructor stub
    }

	/**
	 * @see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse response)
	 */
	protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
		// TODO Auto-generated method stub
		doPost(request,response);
	}

	/**
	 * @see HttpServlet#doPost(HttpServletRequest request, HttpServletResponse response)
	 */
	protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
		// TODO Auto-generated method stubintl
		
	 
		String result="";
		PrintWriter out=response.getWriter();
		try {
			 result= SearchHadoopFile.searchWord(request.getParameter("word"));
		} catch (Exception e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		out.write(result);
		
		out.flush();
		out.close();
	}

}

 

web页面：

写道
<%@ page language="java" contentType="text/html; charset=UTF-8"
pageEncoding="UTF-8"%>
<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>hadoop lucene test</title>
<script type="text/javascript" src="js/jquery-1.7.1.min.js"></script>
<script type="text/javascript">

function queryWord(){
var word=$('#searchWord_id').val();
$.ajax({
type: "POST",
url: "hServlet",
data: { word: word }
}).done(function( msg ) {
$('#count_id').text('结果:'+msg);
});
}
</script>
</head>
<body>
<div>关键字:Hello,World bye,google</div>
查找字段:<input id="searchWord_id" type="text"><input type="button" onclick="queryWord()" value="查找">
<div id="count_id"></div>
</body>
</html>

 

效果界面如下:

新增对这一方案的部分实现：多语言版基础管理系统展示[es6版]-简单整合spring+lucene+hadoop 

 

 

  • 大小: 42.6 KB
  • 大小: 8.4 KB
分享到：
| uml语言基础笔之用例图,组件图,泳道图等
评论
5 楼 minn84 2014-06-20  
demo的思路很简单，主要是两点：
1.hadoop负责处理数据
2.web应用负责展示数据
4 楼 wen262856298 2014-06-03  
重点没有晒啊
3 楼 di1984HIT 2014-05-22  
哪有分布式的啊。晕了。
2 楼 minn84 2014-03-22  
nurenok 写道
是不是利用HDFS

hadoop里面其中就有个概念 hdfs
1 楼 nurenok 2014-03-11  
是不是利用HDFS
发表评论

您还没有登录,请您登录后再发表评论

相关推荐

Magicbox
Global site tag (gtag.js) - Google Analytics