Handle Big Dataset with Real Pagination with Primefaces 3.3 LazyDataModel

If you have millions of records in your database, you would like to use real pagination to provide viewing of the data. By "real" (or true or proper, whatever) pagination, it means that the searching function only fetches a few records for viewing on the current page, and the count of the total records that matche the searching criteria.

 

Primefaces 3.3 has built-in support for this real pagination. To use this mechanisam, the application needs to implement "org.primefaces.model.LazyDataModel " that wraps the data loading and paging etc required for the pagination.

 

Since the backing bean does not cache anything, it could be session scoped safely. As a matter of fact, it would not be working if request scoped. @ViewScoped? don't know.

 

To enable sorting as well, the application needs to provide a java.util.Comparator and use java.util.Collections to sort the loaded list.

 

Let's first take a look at the front end jsf page "/student/studentSearch.xhtml":

 

<ui:composition xmlns="http://www.w3.org/1999/xhtml"  
                    xmlns:h="http://java.sun.com/jsf/html"  
                    xmlns:f="http://java.sun.com/jsf/core"  
                    xmlns:ui="http://java.sun.com/jsf/facelets"  
                    xmlns:p="http://primefaces.org/ui"  
                    template="/template/template1.xhtml">  
      
<ui:define name="title">#{msgs.studentSearch}</ui:define>  
          
<ui:define name="content">  
<h:form id="searchForm">  
	<p:panel header="#{msgs.studentSearch}">  
		<h:panelGrid columns="3">  
		<h:outputLabel value="#{msgs.name}: "/>  
		<h:inputText value="#{ssp.nameFilter}"></h:inputText>  
		<h:commandButton type="submit" value="#{msgs.search}" 
									  action="#{ssp.findByName}"/>  
		</h:panelGrid>  
	</p:panel>  
</h:form>  

<p:panel header="#{msgs.studentList}">  
	<h:form id="dataListForm">  
	<!-- search result list -->  
	<p:dataTable id="idStuDataTable" 
			var="st" value="#{ssp.lazyModel}" lazy="true"  
			paginator="true" rows="5" rowsPerPageTemplate="5,10,20"
			paginatorPosition="bottom"
			paginatorTemplate="{CurrentPageReport} {FirstPageLink} {PreviousPageLink}
						{PageLinks} {NextPageLink} {LastPageLink} {RowsPerPageDropdown}">
							
			<f:facet name="header">
				#{msgs.studentFound}: #{ssp.lazyModel.rowCount}
			</f:facet>

			<p:column headerText="#{msgs.name}" sortBy="#{st.name}">
				<h:outputText value="#{st.name}" />
			</p:column>

			<p:column headerText="#{msgs.mobile}" sortBy="#{st.mobile}">
				<h:outputText value="#{st.mobile}"/>
			</p:column>

			......

	</p:dataTable>
	</h:form>
</p:panel>
      
......   

 

U can see the primefaces data table now gets data from variable "lazyModel" in the backing bean.

 

Here's the source code of the backing bean:

package com.jxee.action.student;

import javax.ejb.EJB;
import javax.faces.bean.ManagedBean;
import javax.faces.bean.RequestScoped;
import javax.faces.bean.SessionScoped;
import javax.faces.bean.ViewScoped;

import org.apache.log4j.Logger;
import org.primefaces.model.LazyDataModel;

import com.jxee.action.ActionBase;
import com.jxee.action.pagination.PmfLazyDataModel;
import com.jxee.ejb.student.StudentDAO;
import com.jxee.model.student.Student;

/**
 * <li>
 * this search bean implements so called "real/true" pagination, 
 * in that it would calculate the total number of records based on the query but
 * only retrieve the necessary records for the page to display.
 * </li>
 * <li>
 * this virtually means that any clicking on sorting and a page number, would result
 * in a query issued to the database to retrieve data.
 * </li>
 * <li>
 * good thing is that you don't need to cache any data
 * </li>
 * <li>
 * it would use Primefaces support for pagination
 * </li>  
 */
@ManagedBean(name="ssp")
@SessionScoped
public class StudentSearchPagination extends ActionBase {

  private static final Logger log = Logger.getLogger(StudentSearchPagination.class);
  private static final String SEARCH_PAGE = "/student/studentSearch.xhtml";
  
  private LazyDataModel<Student> lazyModel;
  private @EJB StudentDAO stDao;

  private String nameFilter;
  
    
  public String findByName() {
    log.debug("Search student by nameFilter: " + nameFilter);
    this.lazyModel = new StudentLazyDataModel(this.stDao, this.nameFilter);
    return SEARCH_PAGE;
  }
  
  public String getNameFilter() {
    return nameFilter;
  }

  public void setNameFilter(String afilter) {
    this.nameFilter = afilter;
  }

  public LazyDataModel<Student> getLazyModel() {
    return lazyModel;
  }

  public void setLazyModel(LazyDataModel<Student> lazyModel) {
    this.lazyModel = lazyModel;
  }

}

  

It becomes simpler compared to previous implementations, since the data retrieving and etc is now handled in the Primefaces LazyDataModel. We need to implement it and here it is:

 

package com.jxee.action.pagination;

import java.util.Collections;
import java.util.List;
import java.util.Map;

import org.apache.log4j.Logger;
import org.primefaces.model.LazyDataModel;
import org.primefaces.model.SortOrder;

import com.jxee.ejb.student.StudentDAO;
import com.jxee.model.student.Student;


public class StudentLazyDataModel extends LazyDataModel<Student> {
  
  private static final Logger log = Logger.getLogger(StudentLazyDataModel.class);
  
  private List<Student> data;
  private StudentDAO dao;
  private String filter;
  
  public StudentLazyDataModel(StudentDAO dao, String filter) {
    this.dao = dao;
    this.filter = filter;
    log.debug("lazy data model inited");
  }
    
  @Override
  public List<Student> load(int startAt, int maxPerPage, String sortField, SortOrder sortOrder, Map<String,String> filters) {
    
    log.debug("Lazy data model loading data...");
    
    try {
      log.debug(String.format("Searching for filter: %s, startAt=%s, maxPerPage=%s", filter, startAt, maxPerPage));
      data = this.dao.findStudent(filter, startAt, maxPerPage);
    }
    catch(Exception e) {
      e.printStackTrace();
    }
    
    // if sorting
    if(sortField != null) {
      log.debug(">>> sortField: " + sortField);
      log.debug(">>> sortOrder: " + sortOrder.toString());
      Collections.sort(data, new StudentLazySorter(sortField, sortOrder));  
    }  
    
    // set the total number of students that have been found
    if(super.getRowCount() <= 0){
      Long total = dao.getFindTotal(filter);
      this.setRowCount(total.intValue());
    }
    
    // set the page size
    this.setPageSize(maxPerPage);
    
    log.debug("Total record match searching criteria: " + getRowCount());
    
    return data;
  }
}

 

 

Here's our sorting comparator, it uses java reflection api to get the values of the sorting fields for comparison:

 

 

package com.jxee.action.pagination;

import java.lang.reflect.Field;
import java.util.Comparator;

import org.apache.log4j.Logger;
import org.primefaces.model.SortOrder;

import com.jxee.model.student.Student;


public class StudentLazySorter implements Comparator<Student> {

  private static final Logger log = Logger.getLogger(LazySorter.class);
  
  private String sortField;
  private SortOrder sortOrder;
  
  
  public LazySorter(String sortField, SortOrder sortOrder) {
    this.sortField = sortField;
    this.sortOrder = sortOrder;
  }

  public int compare(Student t1, Student t2) {
    try {
      Field f = Student.class.getDeclaredField(this.sortField);
      f.setAccessible(true);
      
      log.debug(">>> compare field: " + f.getName());

      Object value1 = f.get(t1);
      Object value2 = f.get(t2);

      int cmp = ((Comparable)value1).compareTo(value2);
      
      return SortOrder.ASCENDING.equals(sortOrder) ? cmp : -1 * cmp;
    }
    catch(Exception e) {
      throw new RuntimeException(e);
    }
  }
}

 

Here's the two methods form the StudentDAO:

 

public List<Student> findStudent(String nameFilter, int start, int max) {
    String filter = nameFilter != null ? "%" + nameFilter + "%" : "%";
    String sql = "select s from Student s where s.name like :pattern order by s.name";
    Query q = em.createQuery(sql);
    q.setParameter("pattern", filter);
    q.setFirstResult(start);  // set start index to retrieve (the page number)
    q.setMaxResults(max);  // set amount of records to retrieve (rows per page)
    return q.getResultList();
}
  
public Long getFindTotal(String nameFilter) {
    String filter = nameFilter != null ? "%" + nameFilter + "%" : "%";
    String sql = "select count(s) from Student s where s.name like :pattern";
    Query q = em.createQuery(sql);
    q.setParameter("pattern", filter);
    return (Long)q.getSingleResult();
}

 

OK, Let take a look at the screen shot:

 

 

Drawbacks:

every click on the page number, or sorting column header, would results a database query executed. it shifts the durden from your application and the application server to database.  but it only retrieves a controlled number of records, say 5/10/20, so it's not a big deal for the database.

 

Further dev:

you can generalize the implementation by java generics.