SSH + Lucene + 分页 + 排序 + 高亮 模拟简单新闻网站搜索引擎

前两天看到了一个中国新闻网，这个网站的搜索form的action是

http://search.chinanews.com/search.do

便知道是struts1的产物，现在都用struts2了，所以给自己的任务是实现Struts2 SSH分页浏览新闻、Lucene分页高亮排序搜索新闻这个两个功能。



    IDE使用的MyEclipse6.5，数据库使用MySQL 5.0.37 , 另装了Navicat for MySQL , jdk版本是6.0

    工程做完的效果图如下，com.zly.indexManager中两个类，分别创建索引和搜索索引，

    com.zly.test.entity中是使用的实体类，分别是NewsType(新闻类型),NewsItem(新闻具体条目),PageControl(分页实体bean) , SearchResultBean(保存搜索结果的bean).

 

   



        浏览和搜索的前提是有据可查，没有数据什么都实现不了 ， 我使用了Htmlparser通过抓取页面信息的形式将新闻添加进数据库 , 添加数据库数据使用了hibernate3

        使用了Annotation的方式完成数据库的映射。

         //NewsType(新闻类型)
Java代码  收藏代码

    package com.zly.test.entity; 
     
    import java.io.Serializable; 
     
    import javax.persistence.Column; 
    import javax.persistence.Entity; 
    import javax.persistence.GeneratedValue; 
    import javax.persistence.GenerationType; 
    import javax.persistence.Id; 
    import javax.persistence.Table; 
    @Entity 
    @Table(name = "t_newsType") 
    public class NewsType implements Serializable { 
     
         
        private static final long serialVersionUID = -5092622762025999122L; 
         
        private Integer id; 
         
        private String newsTypeName; 
        @Column 
        public String getNewsTypeName() { 
            return newsTypeName; 
        } 
     
        public void setNewsTypeName(String newsTypeName) { 
            this.newsTypeName = newsTypeName; 
        } 
        @Id 
        @GeneratedValue(strategy = GenerationType.AUTO) 
        public Integer getId() { 
            return id; 
        } 
     
        public void setId(Integer id) { 
            this.id = id; 
        } 
         
         
    } 



   //NewsItem(新闻具体条目)

 
Java代码  收藏代码

    package com.zly.test.entity; 
     
    import java.io.Serializable; 
    import java.util.Date; 
     
    import javax.persistence.Column; 
    import javax.persistence.Entity; 
    import javax.persistence.GeneratedValue; 
    import javax.persistence.GenerationType; 
    import javax.persistence.Id; 
    import javax.persistence.JoinColumn; 
    import javax.persistence.ManyToOne; 
    import javax.persistence.Table; 
    import javax.persistence.Temporal; 
    import javax.persistence.TemporalType; 
    @Entity 
    @Table(name = "t_newsItem") 
    public class NewsItem implements Serializable { 
     
         
        private static final long serialVersionUID = -7532888546907495633L; 
         
        private Integer id ;  
         
        private String newsTitle ; 
         
        private String newsContent; 
         
        private Date publishTime; 
         
        private String resource; 
         
        private NewsType type; 
         
        private String editor; 
        @Column 
        public String getEditor() { 
            return editor; 
        } 
     
        public void setEditor(String editor) { 
            this.editor = editor; 
        } 
        @ManyToOne 
        @JoinColumn(name = "t_newsType_id") 
        public NewsType getType() { 
            return type; 
        } 
     
        public void setType(NewsType type) { 
            this.type = type; 
        } 
        @Id 
        @GeneratedValue(strategy = GenerationType.AUTO) 
        public Integer getId() { 
            return id; 
        } 
     
        public void setId(Integer id) { 
            this.id = id; 
        } 
        @Column 
        public String getNewsTitle() { 
            return newsTitle; 
        } 
     
        public void setNewsTitle(String newsTitle) { 
            this.newsTitle = newsTitle; 
        } 
         
         
        @Temporal(value = TemporalType.TIMESTAMP) 
        public Date getPublishTime() { 
            return publishTime; 
        } 
     
        public void setPublishTime(Date publishTime) { 
            this.publishTime = publishTime; 
        } 
     
        public String getResource() { 
            return resource; 
        } 
     
        public void setResource(String resource) { 
            this.resource = resource; 
        } 
     
        public String getNewsContent() { 
            return newsContent; 
        } 
     
        public void setNewsContent(String newsContent) { 
            this.newsContent = newsContent; 
        } 
         
         
    } 



    添加所有新闻类型的类放在了com.zly.test.newsFetch包下 ， 具体代码：




Java代码  收藏代码

    package com.zly.test.newsfetch; 
     
     
    import java.text.DateFormat; 
    import java.text.SimpleDateFormat; 
    import java.util.Date; 
    import java.util.LinkedHashSet; 
    import java.util.Set; 
    import java.util.regex.Matcher; 
    import java.util.regex.Pattern; 
     
    import org.hibernate.Session; 
    import org.hibernate.SessionFactory; 
    import org.hibernate.cfg.AnnotationConfiguration; 
    import org.hibernate.cfg.Configuration; 
    import org.htmlparser.Parser; 
    import org.htmlparser.Tag; 
    import org.htmlparser.filters.NodeClassFilter; 
    import org.htmlparser.tags.Div; 
    import org.htmlparser.tags.LinkTag; 
    import org.htmlparser.util.NodeList; 
    import org.htmlparser.visitors.NodeVisitor; 
     
    import com.zly.test.entity.NewsItem; 
    import com.zly.test.entity.NewsType; 
     
    public class GetNews { 
        public static void main(String[] args) throws Exception { 
            //插入数据新闻类型 
            //insertAllTypes(); 
             
            //插入所有新闻数据 
             
            //国内新闻 
            //insertNewsItems("http://www.chinanews.com.cn/scroll-news/gn/2009/05" ,"/news.shtml" ,1); 
            //国际新闻 
            //insertNewsItems("http://www.chinanews.com.cn/scroll-news/gj/2009/05" ,"/news.shtml" ,2); 
            //社会新闻 
            //insertNewsItems("http://www.chinanews.com.cn/scroll-news/sh/2009/05" ,"/news.shtml" ,3); 
            //港澳新闻 
            //insertNewsItems("http://www.chinanews.com.cn/scroll-news/ga/2009/05" ,"/news.shtml" ,4);       
            //台湾新闻 
            //insertNewsItems("http://www.chinanews.com.cn/scroll-news/tw/2009/05" ,"/news.shtml" ,5);       
            //华人新闻 
            //insertNewsItems("http://www.chinanews.com.cn/scroll-news/hr/2009/05" ,"/news.shtml" ,6);       
            //经济新闻 
            //insertNewsItems("http://www.chinanews.com.cn/scroll-news/cj/2009/05" ,"/news.shtml" ,7);       
            //文化新闻 
            //insertNewsItems("http://www.chinanews.com.cn/scroll-news/wh/2009/05" ,"/news.shtml" ,8);       
            //娱乐新闻 
            //insertNewsItems("http://www.chinanews.com.cn/scroll-news/yl/2009/05" ,"/news.shtml" ,9);       
            //体育新闻 
            //insertNewsItems("http://www.chinanews.com.cn/scroll-news/ty/2009/05" ,"/news.shtml" ,10);      
            //教育新闻 
            //insertNewsItems("http://www.chinanews.com.cn/scroll-news/edu/2009/05" ,"/news.shtml" ,11);         
            //健康新闻 
            //insertNewsItems("http://www.chinanews.com.cn/scroll-news/jk/2009/05" ,"/news.shtml" ,12);      
            //生活新闻 
            //insertNewsItems("http://www.chinanews.com.cn/scroll-news/life/2009/05" ,"/news.shtml" ,13);        
            //IT新闻 
            //insertNewsItems("http://www.chinanews.com.cn/scroll-news/it/2009/05" ,"/news.shtml" ,14);      
        } 
        public static void insertAllTypes() { 
            Configuration cfg = new AnnotationConfiguration().configure(); 
            SessionFactory factory = cfg.buildSessionFactory(); 
            Session session = factory.openSession(); 
            String str = new String("国内 国际 社会 港澳 台湾 华人 经济 文化 娱乐 体育 教育 健康 生活 IT"); 
            String[] typesStr = str.split(" "); 
            NewsType[] types = new NewsType[typesStr.length]; 
            session.beginTransaction(); 
            for (int i = 0; i < typesStr.length; i++) { 
                types[i] = new NewsType(); 
                types[i].setNewsTypeName(typesStr[i]); 
                session.save(types[i]); 
            } 
            session.getTransaction().commit(); 
            session.close(); 
        } 
         
        //处理5.1 - 5.5 所有的具体类型的新闻 
         
        public static void insertNewsItems(String before , String after , int type) throws Exception { 
            Configuration cfg = new AnnotationConfiguration().configure(); 
            SessionFactory factory = cfg.buildSessionFactory(); 
            Session session = factory.openSession(); 
            //得到当前新闻所属类别 
            NewsType newsType = (NewsType) session.get(NewsType.class, type); 
            final Set<NewsItem> set = new LinkedHashSet<NewsItem>(); 
            //获取5月1日-5月5日的新闻 
            for (int i = 1; i <= 5; i++) { 
                String src = before; 
                if(i < 10) { 
                    src = src + "0" + i; 
                }else { 
                    src = src + i ;  
                } 
                src = src + after; 
                //使用htmlParser获取每一条新闻的超链接 
                Parser parser = new Parser(src); 
                parser.setEncoding("gb2312"); 
                NodeList divList = parser.parse(new NodeClassFilter(Div.class)); 
                for (int j = 0; j < divList.size(); j++) { 
                    Div div = (Div) divList.elementAt(j); 
                    String divClass = div.getAttribute("id"); 
                    //取得id为news_list的div 
                    if(divClass != null && divClass.equals("news_list")) { 
                        div.accept(new NodeVisitor() { 
                            //遍历news_list里面的所有超链接 
                            public void visitTag(Tag tag) { 
                                if(tag instanceof LinkTag) { 
                                    String href = ((LinkTag)tag).getLink(); 
                                    if(!href.startsWith("http")) { 
                                        href = "http://www.chinanews.com.cn" + href; 
                                    } 
                                    System.out.println(href); 
                                    //找到超链接，将这个超链接所在的页面进行处理，抓取新闻数据，并将其保存在Set中。 
                                    try{ 
                                        insertOneNews(href , set); 
                                    }catch(Exception e) { 
                                        e.printStackTrace(); 
                                    } 
                                     
                                } 
                            } 
                        }); 
                    } 
                } 
            } 
            //获取新闻完成后，将所有NewsItem添加到数据库 
            session.beginTransaction(); 
            for (NewsItem newsItem : set) { 
                newsItem.setType(newsType); 
                session.save(newsItem); 
            } 
            session.getTransaction().commit(); 
            session.close(); 
        } 
        //处理每一个具体的新闻超链接，得到NewsItem类 
        public static void insertOneNews(String src , Set<NewsItem> set) throws Exception { 
            Parser parser = new Parser(src); 
            parser.setEncoding("gb2312"); 
            NodeList divList = parser.extractAllNodesThatMatch(new NodeClassFilter(Div.class)); 
            NewsItem newsItem = new NewsItem(); 
            String title = ""; 
            Date parse = null; 
            String content = ""; 
            String editor = ""; 
            //遍历网页的div。将制定div里面的信息设置到NewsItem实体类中 
            for (int i = 0; i < divList.size(); i++) { 
                Div div = (Div) divList.elementAt(i); 
                String divString = div.getAttribute("class"); 
                if(divString != null) { 
                    //设置标题 
                    if(divString.equals("left_bt")) { 
                        title = div.toPlainTextString(); 
                    } 
                    //设置发布时间 
                    if(divString.equals("left_time")) { 
                        String publishStr = ""; 
                        Pattern pattern = Pattern.compile("[\\d]{4}年[\\d]{2}月[\\d]{2}日 [\\d]{2}:[\\d]{2}"); 
                        Matcher matcher = pattern.matcher(div.toPlainTextString()) ; 
                        if(matcher.find()) { 
                            publishStr = matcher.group(); 
                        } 
                        DateFormat format = new SimpleDateFormat("yyyy年MM月dd日 HH:mm"); 
                        parse = format.parse(publishStr); 
                    } 
                    //设置正文内容 
                    if(divString.equals("left_zw")) { 
                        content = div.toHtml(); 
                    } 
                    //设置记者名称 
                    if(divString.equals("left_name")) { 
                        editor = div.toPlainTextString().trim(); 
                        editor = editor.substring(editor.indexOf(":") + 1, editor.lastIndexOf("】")); 
                    } 
                } 
                 
            } 
            newsItem.setEditor(editor); 
            newsItem.setNewsContent(content); 
            newsItem.setNewsTitle(title); 
            newsItem.setPublishTime(parse); 
            //设置新闻来源 
            newsItem.setResource("最快新闻网"); 
            //将新闻添加到set中 
            set.add(newsItem); 
        } 
    } 



通过上面的代码完成了所有的数据添加工作。



下面根据ssh的流程分别定制dao , manager , action



com.zly.test.dao包中是所有操作dao的抽象类和接口



我们直接看这些接口的实现



//NewsItemDaoHibernate  新闻实体类dao






Java代码  收藏代码

    @SuppressWarnings("unchecked") 
        //根据分页得到具体页的内容 
        public List<com.zly.test.entity.NewsItem> getNewsItemByFirstResultAndMaxResult( 
                final int firstResult, final int maxResult , final int type) { 
            return (List<NewsItem>) this.getHibernateTemplate().execute(new HibernateCallback() { 
                public Object doInHibernate(Session session) 
                        throws HibernateException, SQLException { 
                    return session.createQuery(" from NewsItem where type.id = '" + type + "'").setFirstResult(firstResult).setMaxResults(maxResult).list(); 
                } 
            }); 
        } 
        //得到数据总条数 ， 以便计算总页数 
        public Long getItemCount(final int id) { 
            return (Long) this.getHibernateTemplate().execute(new HibernateCallback(){ 
                public Object doInHibernate(Session session) 
                        throws HibernateException, SQLException { 
                    return session.createQuery("select count(*) from NewsItem where type.id = '" + id + "'").uniqueResult(); 
                } 
            }); 
        } 
        //显示新闻数据页面用到， 查询具体新闻属于哪个新闻类别 
        public int getNewsType(final int id) { 
            return (Integer) this.getHibernateTemplate().execute(new HibernateCallback() { 
                public Object doInHibernate(Session session) 
                        throws HibernateException, SQLException { 
                    NewsItem newsItem = get(id); 
                    NewsType type = newsItem.getType(); 
                    return type.getId(); 
                } 
            }); 
        } 



   只用到了一个   NewsManagerImpl




Java代码  收藏代码

    package com.zly.test.service.impl; 
     
    import java.util.List; 
     
    import com.zly.test.dao.NewsItemDao; 
    import com.zly.test.entity.NewsItem; 
    import com.zly.test.service.NewsManager; 
     
    public class NewsManagerImpl implements NewsManager { 
        //新闻条目dao 
        private NewsItemDao newsItemDao; 
         
        public NewsItemDao getNewsItemDao() { 
            return newsItemDao; 
        } 
     
        public void setNewsItemDao(NewsItemDao newsItemDao) { 
            this.newsItemDao = newsItemDao; 
        } 
        //根据分页得到内容 
        public List<NewsItem> getNewsItemByFirstResultAndMaxResult(int firstResult, 
                int maxResult , int type) { 
            return newsItemDao.getNewsItemByFirstResultAndMaxResult(firstResult, maxResult ,  type); 
        } 
        //得到总记录数，以便计算总页数 
        public Long getItemCounts(int id) { 
            return newsItemDao.getItemCount(id); 
        } 
        //通过id得到具体新闻 
        public NewsItem getNewsById(int id) { 
            return newsItemDao.get(id); 
        } 
        //通过id得到所属新闻类别的id 
        public int getNewsType(int id) { 
            return newsItemDao.getNewsType(id);  
        } 
     
    } 





在applicationContext-action.xml中配置action类别
Java代码  收藏代码

    <bean class="com.zly.test.action.NewsAction" id="newsAction" scope="request"> 
            <property name="newsManager" ref="newsManager"></property> 
            <property name="map" ref="map"></property> 
            <property name="map1" ref="map1"></property> 
    </bean> 



其中定一个两个map , 因为主页的查看分类新闻的url是采用的这种形式<a href="newsAction.action?category=china" target="_blank">国内</a>   名字为map的Map中保存信息如下


Java代码  收藏代码

    <bean id="map" class="java.util.HashMap"> 
       <constructor-arg> 
        <map> 
        <entry key="china" value="1###国内新闻" /> 
            <entry key="world" value="2###国际新闻" /> 
            <entry key="society" value="3###社会新闻" /> 
            <entry key="compatriot" value="4###港台新闻" /> 
            <entry key="taiwan" value="5###台湾新闻" /> 
            <entry key="huaren" value="6###华人新闻" /> 
            <entry key="economic" value="7###经济新闻" /> 
            <entry key="wenhua" value="8###文化新闻" /> 
            <entry key="entertainment" value="9###娱乐新闻" /> 
            <entry key="sports" value="10###体育新闻" /> 
            <entry key="jiaoyu" value="11###教育新闻" /> 
            <entry key="jiankang" value="12###健康新闻" /> 
            <entry key="life" value="13###生活新闻" /> 
            <entry key="keji" value="14###科技新闻" /> 
        </map> 
       </constructor-arg> 
    </bean> 



  key是?category后面的值 ， value是两部分 ， 被###分割开 ， 前面的数值是所属新闻类别的id值， 后面的文字是其类别的文字。将其保存在map中，避免不停地查询数据库。



  分页类PageControl的代码如下：


Java代码  收藏代码

    package com.zly.test.entity; 
     
    import java.util.List; 
     
     
     
     
    public class PageControl { 
         
        private int curPage ; //当前是第几页  
         
        private int maxPage ; //一共有多少页  
         
        private Long maxRowCount ; //一共有多少行  
         
        private int rowsPerPage = 8 ; //每页有多少行  
     
        private List<?> data;//每页的User 
         
        public int getCurPage() { 
            return curPage; 
        } 
     
        public void setCurPage(int curPage) { 
            this.curPage = curPage; 
        } 
     
        public int getMaxPage() { 
            return maxPage; 
        } 
     
        public void setMaxPage(int maxPage) { 
            this.maxPage = maxPage; 
        } 
     
        public List<?> getUserList() { 
            return data; 
        } 
     
        public void setUserList(List<?> data) { 
            this.data = data; 
        } 
     
        public Long getMaxRowCount() { 
            return maxRowCount; 
        } 
     
        public void setMaxRowCount(Long amaxRowCountxRowCount) { 
            this.maxRowCount = amaxRowCountxRowCount; 
        } 
     
        public int getRowsPerPage() { 
            return rowsPerPage; 
        } 
     
        public void setRowsPerPage(int rowsPerPage) { 
            this.rowsPerPage = rowsPerPage; 
        } 
         
        public void countMaxPage() {   //根据总行数计算总页数   
            if (this.maxRowCount % this.rowsPerPage ==0){ 
               this.maxPage = (int) (this.maxRowCount/this.rowsPerPage); 
            }else{ 
               this.maxPage = (int) (this.maxRowCount/this.rowsPerPage + 1);         
            } 
        } 
         
    } 



  被许多页所包含的page.jsp代码如下：




Java代码  收藏代码

    <%@ page language="java"  contentType="text/html;charset=utf-8" 
        pageEncoding="utf-8"%> 
    <%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %> 
    <html> 
    <head> 
        <script language="javascript"> 
            function jumping() { 
                document.pageForm.submit(); 
                return true; 
            } 
            function gotoPage(pageNum) { 
                document.pageForm.jumpPage.value = pageNum ; 
                document.pageForm.submit(); 
                return true; 
            } 
        </script> 
    </head> 
    <body> 
        <c:if test="${pageControl.maxPage != 1}"> 
            <form method="post" action="pageAction.action" name="pageForm"> 
     
            <table> 
                <tr> 
                    <td> 
                        每页${pageControl.rowsPerPage}行 
                    </td> 
                    <td> 
                        共${pageControl.maxRowCount }行 
                    </td> 
                    <td> 
                        第${pageControl.curPage }页 
                    </td> 
                    <td> 
                        共${pageControl.maxPage }页 
                    </td> 
                    <td> 
                        <c:choose> 
                            <c:when test="${pageControl.curPage == 1}"> 
                                首页 上一页 
                            </c:when> 
                             
                            <c:otherwise> 
                                <A HREF="javascript:gotoPage(1)">首页</A> 
                                <A HREF="javascript:gotoPage(${pageControl.curPage - 1} )">上一页</A>   
                            </c:otherwise> 
                        </c:choose> 
                         
                         
                        <c:choose> 
                            <c:when test="${pageControl.curPage == pageControl.maxPage}"> 
                                下一页 尾页 
                            </c:when> 
                             
                            <c:otherwise> 
                                <A HREF="javascript:gotoPage(${pageControl.curPage + 1})">下一页</A> 
                                <A HREF="javascript:gotoPage(${pageControl.maxPage })">尾页</A> 
                            </c:otherwise> 
                        </c:choose> 
                        转到第 
                        <SELECT name="jumpPage" onchange="jumping()"> 
                                    <c:forEach var="i" begin="1" end="${pageControl.maxPage}" step="1"> 
                                        <c:choose> 
                                            <c:when test="${i == pageControl.curPage}"> 
                                                <OPTION selected value="${i}">${i }</OPTION> 
                                            </c:when> 
                                            <c:otherwise> 
                                                <OPTION value="${i}">${i}</OPTION>                   
                                            </c:otherwise> 
                                        </c:choose> 
                                    </c:forEach> 
                        </SELECT> 
                        页 
                    </td> 
                </tr> 
            </table> 
     
        </form> 
        </c:if> 
         
    </body> 
    </html> 



   下面是struts.xml中关于页面展示新闻的配置



 
Java代码  收藏代码

    <action name="newsAction" class="newsAction" > 
        <result type="redirect">pageAction.action</result>                   
    </action> 
     
    <action name="pageAction" class="newsAction" method="pageAction"> 
        <result>/result.jsp</result> 
    </action> 
     
    <action name="detailAction" class="newsAction" method="showDetail"> 
        <result>/detail.jsp</result> 
    </action> 

 

   NewsAction代码如下：

 
Java代码  收藏代码

    package com.zly.test.action; 
     
    import java.util.List; 
    import java.util.Map; 
     
    import com.zly.test.entity.NewsItem; 
    import com.zly.test.entity.PageControl; 
    import com.zly.test.service.NewsManager; 
     
    public class NewsAction extends BaseAction { 
     
        private static final long serialVersionUID = 7780804627621048756L; 
     
        //对应分页jsp中的跳转到第几页 
        private String jumpPage; 
        //对应url?category的值 
        private String category; 
        //对新闻数据进行处理 
        private NewsManager newsManager; 
        //保存<entry key="china" value="1###国内新闻" />这样的信息 
        private Map<String , String> map; 
        //保存<entry key="1" value="china###国内新闻" />这样的信息 
        private Map<String , String> map1; 
         
        public Map<String, String> getMap1() { 
            return map1; 
        } 
     
     
        public void setMap1(Map<String, String> map1) { 
            this.map1 = map1; 
        } 
     
     
        public Map<String, String> getMap() { 
            return map; 
        } 
     
     
        public void setMap(Map<String, String> map) { 
            this.map = map; 
        } 
     
     
        public String getJumpPage() { 
            return jumpPage; 
        } 
     
     
        public void setJumpPage(String jumpPage) { 
            this.jumpPage = jumpPage; 
        } 
     
     
         
        public NewsManager getNewsManager() { 
            return newsManager; 
        } 
     
     
        public void setNewsManager(NewsManager newsManager) { 
            this.newsManager = newsManager; 
        } 
     
        //处理newsAction.action?category=xxx 
        public String execute() { 
            //得到相应的    1###国内新闻 
            String typeIdInfo = map.get(category); 
            //这里得到的是新闻类别id 
            Integer typeId = Integer.parseInt(typeIdInfo.split("###")[0]); 
            //这里得到的是新闻类别字符串 
            String typeString = typeIdInfo.split("###")[1]; 
            //将上面三个变量保存在session中，在jsp页面中显示 
            this.getSession().setAttribute("category", category); 
            this.getSession().setAttribute("typeString" , typeString); 
            this.getSession().setAttribute("id", typeId); 
            //跳转到pageAction中，处理分页 
            return SUCCESS; 
        } 
         
        public String pageAction() { 
            //取出新闻类别id 
            Integer id = (Integer) this.getSession().getAttribute("id"); 
            //查看是不是第一次来分页，如果是，当前页设置成第一个 
            if(jumpPage == null) { 
                jumpPage = "1"; 
            } 
            //从request范围内取出PageControl , 如为空，则第一次分页，创建pageControl对象 
            PageControl pageControl  = (PageControl) this.getRequest().getAttribute("pageControl"); 
            if(pageControl == null) { 
                pageControl = new PageControl(); 
                //第一次的时候得到相应新闻类别所对应的记录的总数 
                pageControl.setMaxRowCount(newsManager.getItemCounts(id)); 
                //计算一共多少页 
                pageControl.countMaxPage(); 
            } 
            //设置jumpPage的值为当前页 
            pageControl.setCurPage(Integer.parseInt(jumpPage)); 
            //计算出hibernate中firstResult的值 
            int firstResult = (pageControl.getCurPage() - 1) * pageControl.getRowsPerPage() + 1; 
            //设置hibernate中maxResult的值 
            int maxResult = pageControl.getRowsPerPage(); 
            //利用hibernate得到当前页的数据，并保存在pageControl分页实体类中 
            List<NewsItem> userList = newsManager.getNewsItemByFirstResultAndMaxResult(firstResult, maxResult , id); 
            pageControl.setData(userList); 
            //将分页实体类保存在request范围内。 
            this.getRequest().setAttribute("pageControl", pageControl); 
            //将页面视图跳转到result.jsp 
            return SUCCESS; 
        } 
         
        public String showDetail() { 
            //得到url中的?id=xxx 
            String newsId = this.getRequest().getParameter("id"); 
            int id = Integer.parseInt(newsId); 
            //使用hibernate得到具体id的新闻记录 
            NewsItem item = newsManager.getNewsById(id); 
            //得到id记录所对应的新闻类型的值   map1这种形式<entry key="1" value="china###国内新闻" /> 
            int typeId = newsManager.getNewsType(id); 
            //得到china,添加到jsp页面的<a href="newsAction?category=">里面 
            String category = map1.get("" + typeId).split("###")[0]; 
            //得到国内新闻，显示在jsp页面的多个位置 
            String typeString = map1.get("" + typeId).split("###")[1]; 
            //将以上多个数据添加到request范围内，以便显示。 
            this.getRequest().setAttribute("news", item); 
            this.getRequest().setAttribute("category", category); 
            this.getRequest().setAttribute("typeString", typeString); 
            return SUCCESS; 
        } 
     
     
        public String getCategory() { 
            return category; 
        } 
     
     
        public void setCategory(String category) { 
            this.category = category; 
        } 
         
    } 





首页页面index.jsp，里面有几个分类超链接和搜索对话框


Java代码  收藏代码

    <%@ page language="java"  pageEncoding="utf-8"%> 
     
    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> 
    <html> 
       
      <head> 
        <title>最快新闻网</title> 
        <link href="css/main.css" rel="stylesheet" type="text/css" /> 
      </head> 
       
      <body> 
             <div id="mainlink"> 
                    <span><a href="index.jsp">最快新闻网</a></span><br /><br /> 
                    <a href="newsAction.action?category=china" target="_blank">国内</a> | <a href="newsAction.action?category=world" target="_blank">国际</a> | <a href="newsAction.action?category=society" target="_blank">社会</a> | <a href="newsAction.action?category=compatriot" target="_blank">港澳</a> | <a href="newsAction.action?category=taiwan" target="_blank">台湾</a> | <a href="newsAction.action?category=huaren" target="_blank">华人</a> | <a href="newsAction.action?category=economic" target="_blank">经济</a> | <a href="newsAction.action?category=wenhua" target="_blank">文化</a> | <a href="newsAction.action?category=entertainment" target="_blank">娱乐</a> | <a href="newsAction.action?category=sports" target="_blank">体育</a> | <a href="newsAction.action?category=jiaoyu" target="_blank">教育</a> | <a href="newsAction.action?category=jiankang" target="_blank">健康</a> | <a href="newsAction.action?category=life" target="_blank">生活</a> | <a href="newsAction.action?category=keji" target="_blank">IT</a><br /> 
                    <form action="searchAction.action" method="post" target="_blank"> 
                        本站搜索： 
                        <input type="text" name="searchParam" style="height:20px"/> 
                        <select name="searchWhich"> 
                            <option value="title"/>标题 
                            <option value="content"/>内容 
                        </select> 
                        <input type="submit" value="搜索" style="height:23px"/> 
                    </form>             
             </div> 
      </body> 
    </html> 







  其表现形式如下：

新闻分页展示页面result.jsp代码如下：
Html代码  收藏代码

    <%@ page language="java" contentType="text/html;charset=utf-8" 
        pageEncoding="utf-8"%> 
    <%@taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %> 
    <%@taglib prefix="fmt" uri="http://java.sun.com/jsp/jstl/fmt" %> 
    <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> 
    <html> 
    <head> 
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> 
    <title>${typeString} -- 最快新闻网</title> 
    </head> 
    <body> 
        <jsp:include page="index.jsp"></jsp:include> 
        <div id="content"> 
            <font color="red" size="5">${typeString}</font> 
            <br /><br /><br /><br /> 
            <div id="newsListTitle" style="float:left;"> 
                <c:forEach items="${pageControl.data}" var="news"> 
                    <div style="margin-top: 20px;"> 
                        <span> 
                            <a href="detailAction.action?id=${news.id }">${news.newsTitle }</a> 
                        </span> 
                        <br /> 
                    </div> 
                </c:forEach>       
            </div> 
             
            <div id="newsListTime"> 
                <c:forEach items="${pageControl.data}" var="news"> 
                    <div style="margin-top: 20px;"> 
                        <span> 
                            <fmt:formatDate value="${news.publishTime}" pattern="MM-dd HH:mm"/> 
                        </span> 
                        <br /> 
                    </div> 
                </c:forEach> 
            </div> 
            <br /><br /><br /><br /> 
            <%@ include file="page.jsp" %> 
        </div> 
    </body> 
    </html> 

  显示效果如下：







其中点击具体超链接的效果图如下：







任务1 到此完成，新闻显示工作结束。下面是搜索引擎部分。



搜索的工具类放置在com.zly.indexManager包下面



说明，本程序使用了庖丁解牛中文分词，用户使用时需要中文字典，我的字典放在了c:\dic下面，使用庖丁还需要配置环境变量PAODING_DIC_HOME ， 其值为c:\dic ， (就是你的字典文件所在的目录)



代码如下：



创建索引类IndexCreateUtil


Java代码  收藏代码

    package com.zly.indexManager; 
     
    import java.io.File; 
    import java.text.DateFormat; 
    import java.text.SimpleDateFormat; 
    import java.util.List; 
     
    import net.paoding.analysis.analyzer.PaodingAnalyzer; 
     
    import org.apache.lucene.analysis.Analyzer; 
    import org.apache.lucene.document.Document; 
    import org.apache.lucene.document.Field; 
    import org.apache.lucene.index.IndexWriter; 
    import org.hibernate.SessionFactory; 
    import org.hibernate.cfg.AnnotationConfiguration; 
    import org.hibernate.cfg.Configuration; 
    import org.hibernate.Session; 
    import org.htmlparser.Parser; 
     
    import com.zly.test.entity.NewsItem; 
     
    public class IndexCreateUtil { 
         
        @SuppressWarnings("unchecked") 
        public void createIndexForNews() throws Exception { 
            //存放索引的文件夹 
            File indexFile = new File("c:/index/news"); 
            //使用了庖丁解牛分词器 
            Analyzer analyzer = new PaodingAnalyzer(); 
            //使用索引文件夹，庖丁解牛分词器创建IndexWriter 
            IndexWriter indexWriter = new IndexWriter(indexFile , analyzer , true); 
            //从数据库中读取出所有的新闻记录以便进行索引的创建 
            Configuration cfg = new AnnotationConfiguration().configure(); 
            SessionFactory factory = cfg.buildSessionFactory(); 
            Session session = factory.openSession(); 
            List<NewsItem> list = session.createQuery(" from NewsItem").list(); 
            DateFormat format = new SimpleDateFormat("yyyy年MM月dd日 HH时mm分ss秒"); 
            //对所有的新闻实体进行索引创建 
            for (NewsItem newsItem : list) { 
                //建立一个lucene文档 
                Document doc = new Document(); 
                //得到新闻标题 
                String newsTitle = newsItem.getNewsTitle(); 
                //得到新闻内容 
                String newsContent = newsItem.getNewsContent(); 
                //得到新闻事件 
                String publishDate = format.format(newsItem.getPublishTime()); 
                //得到新闻主键id 
                String id = newsItem.getId() + ""; 
                //将新闻标题加入文档，因为要搜索和高亮，所以index是tokennized，TermVector是WITH_POSITIONS_OFFSETS 
                doc.add(new Field("title" , newsTitle , Field.Store.YES , Field.Index.TOKENIZED , Field.TermVector.WITH_POSITIONS_OFFSETS)); 
                //利用htmlparser得到新闻内容html的纯文本 
                Parser parser = new Parser(); 
                parser.setInputHTML(newsContent); 
                String strings = parser.parse(null).elementAt(0).toPlainTextString().trim(); 
                //添加新闻内容至文档，与标题相似 
                doc.add(new Field("content" , strings , Field.Store.COMPRESS , Field.Index.TOKENIZED , Field.TermVector.WITH_POSITIONS_OFFSETS)); 
                //添加时间至文档，因为要按照此字段降序排列排序，所以tokenzied,不用高亮所以TermVector是no就行了 
                doc.add(new Field("date" , publishDate , Field.Store.YES , Field.Index.TOKENIZED , Field.TermVector.NO)); 
                //添加主键至文档，不分词，不高亮。 
                doc.add(new Field("id" , id , Field.Store.YES , Field.Index.NO , Field.TermVector.NO)); 
                indexWriter.addDocument(doc); 
            } 
            //创建索引 
            indexWriter.optimize(); 
            indexWriter.close(); 
            //关闭session 
            session.close(); 
        } 
        public static void main(String[] args) throws Exception { 
            IndexCreateUtil util  = new IndexCreateUtil(); 
            util.createIndexForNews(); 
        } 
    } 



对索引进行搜索的代码如下：



 
Java代码  收藏代码

    package com.zly.indexManager; 
     
    import java.io.File; 
    import java.util.ArrayList; 
    import java.util.List; 
     
    import net.paoding.analysis.analyzer.PaodingAnalyzer; 
     
    import org.apache.lucene.analysis.Analyzer; 
    import org.apache.lucene.document.Document; 
    import org.apache.lucene.index.IndexReader; 
    import org.apache.lucene.queryParser.QueryParser; 
    import org.apache.lucene.search.Hits; 
    import org.apache.lucene.search.IndexSearcher; 
    import org.apache.lucene.search.Query; 
    import org.apache.lucene.search.Sort; 
    import org.apache.lucene.search.highlight.Highlighter; 
    import org.apache.lucene.search.highlight.QueryScorer; 
    import org.apache.lucene.search.highlight.SimpleFragmenter; 
    import org.apache.lucene.search.highlight.SimpleHTMLFormatter; 
     
    import com.zly.test.entity.SearchResultBean; 
     
    public class IndexSearchUtil { 
         
        public List<SearchResultBean> getSearchResult(String searchWhich , String searchParam , int firstResult , int maxResult) throws Exception{ 
            //索引所在文件夹 
            File indexFile = new File("c:/index/news"); 
            //读取索引的indexReader 
            IndexReader reader = IndexReader.open(indexFile); 
            //庖丁解牛分词器 
            Analyzer analyzer = new PaodingAnalyzer(); 
            //指定对content还是title进行查询 
            QueryParser parser = new QueryParser(searchWhich , analyzer); 
            //创建indexSearcher 
            IndexSearcher searcher  = new IndexSearcher(reader); 
            //对用户的输入进行查询 
            Query query = parser.parse(searchParam); 
            //根据date字段进行排序，得到查询结果 
            Hits hits = searcher.search(query , new Sort("date" , true)); 
            //创建list，将结果保存其中，以便在jsp页面中进行显示 
            List<SearchResultBean> list = new ArrayList<SearchResultBean>(); 
            //模拟hibernate的serFirstResult和setMaxResult以便返回指定条目的结果 
            for (int i = firstResult - 1; i < firstResult + maxResult - 1; i++) { 
                Document doc = hits.doc(i); 
                //取得该条索引文档 
                SearchResultBean srb = new SearchResultBean(); 
                //从中取出标题 
                String title = doc.get("title"); 
                //从中取出内容 
                String content = doc.get("content"); 
                //从中取出主键id 
                String id = doc.get("id"); 
                //从中取出发布时间 
                String date = doc.get("date"); 
                //高亮htmlFormatter对象 
                SimpleHTMLFormatter sHtmlF = new SimpleHTMLFormatter("<b><font color='red'>", "</font></b>"); 
                //高亮对象 
                Highlighter highlighter = new Highlighter(sHtmlF,new QueryScorer(query)); 
                //设置高亮附近的字数 
                highlighter.setTextFragmenter(new SimpleFragmenter(100)); 
                //如果查询的是标题，进行处理 
                if(searchWhich.equals("title")) { 
                    String bestFragment = highlighter.getBestFragment(analyzer,searchWhich,title); 
                    //获得高亮后的标题内容 
                    srb.setTitle(bestFragment); 
                    //如果内容不足150个字，全部设置 
                    if(content.length() < 150) { 
                        srb.setContent(content); 
                    }else { 
                        //如果内容多于150个字，只取出前面150个字 
                        srb.setContent(content.substring(0 , 150)); 
                    } 
                } else { 
                    //如果查询的是内容字段 
                    String bestFragment = highlighter.getBestFragment(analyzer,searchWhich,content); 
                    //取得高亮内容并设置 
                    srb.setContent(bestFragment); 
                    //设置标题，全部设置 
                    srb.setTitle(title); 
                } 
                //设置日期 
                srb.setDate(date); 
                //设置主键 
                srb.setId(id); 
                //添加到list中，以便在jsp页面上显示 
                list.add(srb); 
            } 
            return list; 
        } 
        //取得符合搜索条件的所有记录总数，以便分页 , 与上面方法类似 
        public int getResultCount(String searchWhich , String searchParam) throws Exception { 
            File indexFile = new File("c:/index/news"); 
            IndexReader reader = IndexReader.open(indexFile); 
            Analyzer analyzer = new PaodingAnalyzer(); 
            QueryParser parser = new QueryParser(searchWhich , analyzer); 
            IndexSearcher searcher  = new IndexSearcher(reader); 
            Query query = parser.parse(searchParam); 
            Hits hits = searcher.search(query); 
            return hits.length(); 
        } 
    } 





分页action代码如下：
Java代码  收藏代码

    package com.zly.test.action; 
     
    import java.util.List; 
     
    import com.zly.indexManager.IndexSearchUtil; 
    import com.zly.test.entity.PageControl; 
    import com.zly.test.entity.SearchResultBean; 
     
    public class SearchAction extends BaseAction { 
     
         
        private static final long serialVersionUID = -2387037924517370511L; 
        //查询索引实体类 
        private IndexSearchUtil indexSearcher; 
        //对应搜索字段是标题还是内容 
        private String searchWhich; 
        //对应用户输入的搜索内容 
        private String searchParam; 
        //对应分页跳转到的页面 
        private String jumpPage; 
         
        public String getJumpPage() { 
            return jumpPage; 
        } 
     
        public void setJumpPage(String jumpPage) { 
            this.jumpPage = jumpPage; 
        } 
     
        public String getSearchWhich() { 
            return searchWhich; 
        } 
     
        public void setSearchWhich(String searchWhich) { 
            this.searchWhich = searchWhich; 
        } 
     
        public String getSearchParam() { 
            return searchParam; 
        } 
     
        public void setSearchParam(String searchParam) { 
            this.searchParam = searchParam; 
        } 
     
        public String search() throws Exception { 
            //如果为空，说明第一次进入分页 
            if(jumpPage == null) { 
                jumpPage = "1"; 
            } 
            //从request范围内取得pageControl对象 
            PageControl pageControl  = (PageControl) this.getRequest().getAttribute("pageControl"); 
            //如果为空，则是第一次分页，创建分页对象，并且设置总的记录条数，以便设置最大页数  
            if(pageControl == null) { 
                pageControl = new PageControl(); 
                pageControl.setMaxRowCount((long)indexSearcher.getResultCount(searchWhich, searchParam)); 
                pageControl.countMaxPage(); 
            } 
            //设置当前页 
            pageControl.setCurPage(Integer.parseInt(jumpPage)); 
            //计算firstResult 
            int firstResult = (pageControl.getCurPage() - 1) * pageControl.getRowsPerPage() + 1; 
            //计算从当前条数算还有多少条记录 
            long left = pageControl.getMaxRowCount() - firstResult; 
            int maxResult = -1; 
            //如果剩余的记录数不如每页显示数，就设置maxResult为剩余条数 
            if(left < pageControl.getRowsPerPage()) { 
                maxResult = Integer.valueOf(left + ""); 
            //如果剩余记录数大于每页显示页数，就设置maxResult为每页条数 
            }else { 
                maxResult = pageControl.getRowsPerPage();  
            } 
            //取得查询结果集 
            List<SearchResultBean> userList = indexSearcher.getSearchResult(searchWhich, searchParam, firstResult, maxResult); 
            //设置为pageControl 
            pageControl.setData(userList); 
            //将pageControl设置到request范围，以便在jsp现实结果 
            this.getRequest().setAttribute("pageControl", pageControl); 
            //将searchWhich和searchParam设置到request范围，以便添加到分页jsp的form里面的hidden表单域，以便下次分页时，能够将值提交过来 
            this.getRequest().setAttribute("searchWhich", searchWhich); 
            this.getRequest().setAttribute("searchParam", searchParam); 
            //跳转到分页视图 
            return SUCCESS; 
             
        } 
     
        public IndexSearchUtil getIndexSearcher() { 
            return indexSearcher; 
        } 
     
        public void setIndexSearcher(IndexSearchUtil indexSearcher) { 
            this.indexSearcher = indexSearcher; 
        } 
         
    } 





搜索的action在struts.xml中设置如下：


Xml代码  收藏代码

    <action name="searchAction" class="searchAction" method="search"> 
        <result>/searchResult.jsp</result> 
    </action> 



//searchResult.jsp代码如下：


Html代码  收藏代码

    <%@ page language="java" contentType="text/html;charset=utf-8" 
        pageEncoding="utf-8"%> 
    <%@taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %> 
    <%@taglib prefix="fmt" uri="http://java.sun.com/jsp/jstl/fmt" %> 
    <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> 
    <html> 
    <head> 
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> 
    <title>${searchParam} 的搜查结果 -- 最快新闻网</title> 
    </head> 
    <body> 
        <jsp:include page="index.jsp"></jsp:include> 
        <div id="content"> 
             
            <div id="searchResults" > 
                <c:forEach items="${pageControl.data}" var="result"> 
                    <div style="margin-top: 20px;"> 
                        <span> 
                            <a href="detailAction.action?id=${result.id }">${result.title}</a><br /> 
                            ${result.content } 
                            <font color="green">http://localhost:8080/NewsWithSearch/detailAction.action?id=${result.id } ${result.date }</font> 
                        </span> 
                        <br /> 
                    </div> 
                </c:forEach>       
            </div> 
             
            <br /><br /><br /><br /> 
             
            <%@ include file="searchPage.jsp" %> 
        </div> 
    </body> 
    </html> 

    其运行结果如图所示（按标题搜索）：











按内容搜索的运行结果如下：







至此，本小项目的所有功能完成，虽然没有多少难度，也不是什么高科技， 俺还是在google和javaeye上查了不少资料，总算是做完了，贴出来，与大家分享，也给新手学习提供资料。



所有的资源我都添加到了附件中，学过ssh的同学应该能够成功部署项目并运行。



其中NewsWithSearch.rar是工程文件夹，包含了所有的代码文件和jar包，加压完直接引到MyEclipse里就行，data.rar是所有的sql语句，插入到MySQL之前应先建立数据库mynews  ,     dic.rar是庖丁解牛用到的字典文件，

解压成一个文件夹，并配置环境变量PAODING_DIC_HOME,其值就是你把它解压成的文件夹（例如c:\dic），最后如果你不想创建索引的话，可以把news.rar解压成一个文件夹，拷贝到c:\index\news下面。






搜索引擎和网络爬虫技术群293961767欢迎志同道合的朋友加入！