Spring Batch 之 Sample（复合格式文件的读、多文件的写）(七) -

mowengaobo

浏览: 166181 次
性别:
来自: 武汉

最近访客更多访客>>

卖火柴的小驴子

qindongliang1922

longhua828

bluecrow_1986

博主相关

博客

微博

相册

留言

关于我

文章分类

社区版块

存档分类

Spring Batch 之 Sample（复合格式文件的读、多文件的写）(七)

博客分类：

spring batch

前面关于Spring Batch的文章，讲述了SpringBatch对CSV文件的读写操作、对XML文件的操作，以及对固定长格式文件的操作。这些事例，同一个Reader读取的都是相同格式的数据，最终写入一个文件。如果遇到下面这样的数据，并想将学生信息和商品信息分类后写入两个文件，应该如何处理呢？

student,200001,ZhangSan,18,78
goodsPNH001011000200.1zhangshana2011/12/18 01:12:36
student,200002,LiSi,19,79
goodsPNH001022000300.1zhangshanb2011/12/19 01:12:36
student,200003,WangWu,20,80
goodsPNH001033000400.1zhangshanc2011/12/20 01:12:36

* 以student开头的数据代表学生信息，以goods开头代表商品信息

这次将和大家一起探讨Spring Batch读取复合格式的数据，然后写入不同的文件的处理方式。

工程结构如下图：

applicationContext.xml和log4j.xml前文已经叙述过，在此不做赘述。

本实例的核心配置文件batch.mxl内容如下：

<?xml version="1.0" encoding="UTF-8"?>
 <bean:beans xmlns="http://www.springframework.org/schema/batch"
     xmlns:bean="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xmlns:p="http://www.springframework.org/schema/p" xmlns:tx="http://www.springframework.org/schema/tx"
     xmlns:aop="http://www.springframework.org/schema/aop" xmlns:context="http://www.springframework.org/schema/context"
     xmlns:util="http://www.springframework.org/schema/util"
     xsi:schemaLocation="http://www.springframework.org/schema/beans 
 http://www.springframework.org/schema/beans/spring-beans-3.0.xsd  
 http://www.springframework.org/schema/tx 
 http://www.springframework.org/schema/tx/spring-tx-3.0.xsd  
 http://www.springframework.org/schema/aop 
 http://www.springframework.org/schema/aop/spring-aop-3.0.xsd  
 http://www.springframework.org/schema/context 
 http://www.springframework.org/schema/context/spring-context-2.5.xsd
 http://www.springframework.org/schema/batch 
 http://www.springframework.org/schema/batch/spring-batch-2.1.xsd 
 http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd">
 
     <bean:import resource="applicationContext.xml" />
     <!-- Job的配置信息 -->
     <job id="multiTypeSingleFileJob">
         <step id="xmlFileReadAndWriterStep">
             <tasklet>
                 <chunk reader="multiTypesItemReader" writer="multiTypesItemWriter"
                     commit-interval="1">
                     <streams>
                         <stream ref="studentWriter" />
                         <stream ref="goodsWriter" />
                     </streams>
                 </chunk>
             </tasklet>
         </step>
     </job>
 
     <!-- 不同格式数据的文件读取 -->
     <bean:bean id="multiTypesItemReader"
         class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
         <bean:property name="resource"
             value="file:#{jobParameters['inputFilePath']}" />
         <bean:property name="lineMapper">
             <bean:bean
 class="org.springframework.batch.item.file.mapping.PatternMatchingCompositeLineMapper">
                 <bean:property name="tokenizers">
                     <bean:map>
                         <bean:entry key="student*" value-ref="studentTokenizer" />
                         <bean:entry key="goods*" value-ref="goodsTokenizer" />
                     </bean:map>
                 </bean:property>
                 <bean:property name="fieldSetMappers">
                     <bean:map>
                         <bean:entry key="student*" value-ref="studentFieldSetMapper" />
                         <bean:entry key="goods*" value-ref="goodsFieldSetMapper" />
                     </bean:map>
                 </bean:property>
             </bean:bean>
         </bean:property>
     </bean:bean>
     <bean:bean id="studentTokenizer"
         class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
         <bean:property name="delimiter" value="," />
         <bean:property name="names">
             <bean:list>
                 <bean:value>student</bean:value>
                 <bean:value>ID</bean:value>
                 <bean:value>name</bean:value>
                 <bean:value>age</bean:value>
                 <bean:value>score</bean:value>
             </bean:list>
         </bean:property>
     </bean:bean>
     <bean:bean id="studentFieldSetMapper"
         class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
         <bean:property name="prototypeBeanName" value="student" />
         <bean:property name="distanceLimit" value="100" />
     </bean:bean>
     <!-- 学生Pojo类 -->
     <bean:bean id="student"
         class="com.wanggc.springbatch.sample.multitypessinglefile.pojo.Student"
         scope="prototype" />
 
     <bean:bean id="goodsTokenizer"
         class="org.springframework.batch.item.file.transform.FixedLengthTokenizer">
         <bean:property name="columns" value="6-13,14-17,18-22,23-32,33-" />
         <bean:property name="names"
             value="isin,quantity,price,customer,buyDay" />
     </bean:bean>
     <bean:bean id="goodsFieldSetMapper"
         class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
         <bean:property name="prototypeBeanName" value="goods" />
     </bean:bean>
     <!-- 商品Pojo类 -->
     <bean:bean id="goods"
         class="com.wanggc.springbatch.sample.multitypessinglefile.pojo.Goods"
         scope="prototype" />
 
     <bean:bean id="multiTypesItemWriter"
         class="com.wanggc.springbatch.sample.multitypessinglefile.MultiItemWriter">
         <bean:property name="delegates">
             <bean:list>
                 <bean:ref bean="studentWriter" />
                 <bean:ref bean="goodsWriter" />
             </bean:list>
         </bean:property>
     </bean:bean>
     <!-- 学生信息的写 -->
     <bean:bean id="studentWriter"
         class="org.springframework.batch.item.file.FlatFileItemWriter" scope="step">
         <bean:property name="resource"
             value="file:#{jobParameters['outputFilePathStudent']}" />
         <bean:property name="lineAggregator">
             <bean:bean
 class="org.springframework.batch.item.file.transform.FormatterLineAggregator">
                 <bean:property name="fieldExtractor">
                     <bean:bean
 class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
                         <bean:property name="names" value="ID,name,age,score" />
                     </bean:bean>
                 </bean:property>
                 <bean:property name="format" value="%-9s%-9s%3d%-2.0f" />
             </bean:bean>
         </bean:property>
     </bean:bean>
     <!-- 商品信息的写 -->
     <bean:bean id="goodsWriter"
         class="org.springframework.batch.item.file.FlatFileItemWriter" scope="step">
         <bean:property name="resource"
             value="file:#{jobParameters['outputFilePathGoods']}" />
         <bean:property name="lineAggregator">
             <bean:bean
 class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
                 <bean:property name="fieldExtractor">
                     <bean:bean
 class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
                         <bean:property name="names"
                             value="isin,quantity,price,customer,buyDay" />
                     </bean:bean>
                 </bean:property>
             </bean:bean>
         </bean:property>
     </bean:bean>
 </bean:beans>

21-33行配置了Job的基本信息。

36-57行配置了Reader的基本信息。FlatFileItemReader的lineMapper属性使用SpringBatch核心类PatternMatchingCompositeLineMapper的时候，会将读取的记录按照不同的方式映射成我们的Pojo对象。当然首先我们要配置不同的tokenizers（43-48）和fieldSetMappers（49-54），并告诉它当前的记录按照那条原则去解析和映射。如45行所示，我们指定key为student*的时候，用studentTokenizer去解析成fieldset，用studentFieldSetMapper将studentTokenizer解析好的fieldset记录映射成Student对象。我们指定的key，其实也就是student开头的记录，*是通配符。PatternMatchingCompositeLineMapper支持两种通配符：*和？，前者代表多个字符，后者仅代表一个字符。至于student和goods信息如何映射成pojo对象，前面的文章中已经做过详细的介绍，这里就不做赘述了。

96-104行配置了Writer的基本信息。Writer也是使用代理的方式，学生信息使用106-122行定义的studentWriter按照固定长的格式写入学生信息文件中，商品信息使用124-141行定义的goodsWriter按照CSV的格式写入商品信息文件中。MultiItemWriter的代码很简单，就不做详细解释了。如下：

package com.wanggc.springbatch.sample.multitypessinglefile;

import java.util.ArrayList;
import java.util.List;

import org.springframework.batch.item.ItemWriter;

import com.wanggc.springbatch.sample.multitypessinglefile.pojo.Goods;
import com.wanggc.springbatch.sample.multitypessinglefile.pojo.Student;

/**
 * 写处理类。
 * 
 * @author Wanggc
 * 
 * @param <T>
*/
@SuppressWarnings("unchecked")
public class MultiItemWriter<T> implements ItemWriter<T> {
/** 写代理 */
private List<ItemWriter<? super T>> delegates;

public void setDelegates(List<ItemWriter<? super T>> delegates) {
this.delegates = delegates;
    }

    @Override
public void write(List<? extends T> items) throws Exception {
// 学生信息的Writer
        ItemWriter studentWriter = (ItemWriter) delegates.get(0);
// 商品信息的Writer
        ItemWriter goodsWriter = (ItemWriter) delegates.get(1);
// 学生信息
        List<Student> studentList = new ArrayList<Student>();
// 商品信息
        List<Goods> goodsList = new ArrayList<Goods>();
// 将传过来的信息按照不同的类型添加到不同的List中
        for (int i = 0; i < items.size(); i++) {
if ("Student".equals(items.get(i).getClass().getSimpleName())) {
                studentList.add((Student) items.get(i));
            } else {
                goodsList.add((Goods) items.get(i));
            }
        }
// 如果学生List中有数据，就执行学生信息的写
        if (studentList.size() > 0) {
            studentWriter.write(studentList);
        }
// 如果商品List中有数据，就执行商品信息的写
        if (goodsList.size() > 0) {
            goodsWriter.write(goodsList);
        }
    }
}

至此，复合文件的读写操作已经讨论结束了。注意实例没有配置Processor。下面是一些辅助文件的信息。

student和goods类的信息与前面文章一样，就不再贴出代码了。

Job启动的代码如下：

package com.wanggc.springbatch.sample.multitypessinglefile;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

public class Launch {
public static void main(String[] args) {
        ApplicationContext context = new ClassPathXmlApplicationContext(
                "batch.xml");
        JobLauncher launcher = (JobLauncher) context.getBean("jobLauncher");
        Job job = (Job) context.getBean("multiTypeSingleFileJob");

try {
// JOB实行
            JobExecution result = launcher.run(
                    job,
new JobParametersBuilder()
                            .addString("inputFilePath",
                                    "C:\\testData\\multiTypesInput.txt")
                            .addString("outputFilePathStudent",
                                    "C:\\testData\\student.txt")
                            .addString("outputFilePathGoods",
                                    "C:\\testData\\goods.csv")
                            .toJobParameters());
// 运行结果输出
            System.out.println(result.toString());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}