`
DavyJones2010
  • 浏览: 153780 次
  • 性别: Icon_minigender_1
  • 来自: 杭州
社区版块
存档分类
最新评论

Spring Batch: Intro II -- Read-Write-Process for Derby

阅读更多

1. In the following example, we will read data from multiple csv files and then write to Derby DB.

    1) In order to read from multiple csv, we need "org.springframework.batch.item.file.MultiResourceItemReader" as reader.

        And config flatFileReader for delegate which reading single csv file.

    2) In order to write to Derby, we need "org.springframework.batch.item.database.JdbcBatchItemWriter" as writer.

        And config derbyDataSource as datasource.

 

2.

    1) pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>edu.xmu.spring.batch</groupId>
	<artifactId>spring-batch</artifactId>
	<version>0.0.1-SNAPSHOT</version>

	<properties>
		<spring.version>3.2.2.RELEASE</spring.version>
		<spring.batch.version>2.2.0.RELEASE</spring.batch.version>
		<junit.version>4.11</junit.version>
	</properties>

	<dependencies>
		<!-- Spring Core -->
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-core</artifactId>
			<version>${spring.version}</version>
		</dependency>
		<!-- Spring jdbc, for database -->
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-jdbc</artifactId>
			<version>${spring.version}</version>
		</dependency>
		<!-- Derby database driver -->
		<dependency>
			<groupId>org.apache.derby</groupId>
			<artifactId>derby</artifactId>
			<version>10.10.1.1</version>
		</dependency>
		<!-- Spring Batch dependencies -->
		<dependency>
			<groupId>org.springframework.batch</groupId>
			<artifactId>spring-batch-core</artifactId>
			<version>${spring.batch.version}</version>
		</dependency>
		<dependency>
			<groupId>org.springframework.batch</groupId>
			<artifactId>spring-batch-infrastructure</artifactId>
			<version>${spring.batch.version}</version>
		</dependency>
	</dependencies>
</project>

   2) domain.ddl 

DROP TABLE ROOT.DOMAIN;
CREATE TABLE ROOT.DOMAIN(ID INT NOT NULL, USERNAME VARCHAR(100), PASSWORD VARCHAR(100), DATE_ADDED DATE);

   3) Domain.java

package edu.xmu.spring.batch.model;

import java.util.Date;

public class Domain {
    int id;
    String username;
    String password;
    Date dateAdded;

    public int getId() {
	return id;
    }

    public void setId(int id) {
	this.id = id;
    }

    public String getUsername() {
	return username;
    }

    public void setUsername(String username) {
	this.username = username;
    }

    public String getPassword() {
	return password;
    }

    public void setPassword(String password) {
	this.password = password;
    }

    public Date getDateAdded() {
	return dateAdded;
    }

    public void setDateAdded(Date dateAdded) {
	this.dateAdded = dateAdded;
    }

}

    4) context.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.springframework.org/schema/beans 
		http://www.springframework.org/schema/beans/spring-beans-3.2.xsd">

	<import resource="database.xml" />
	<import resource="../jobs/spring-batch-job-derby.xml" />

	<bean id="jobLauncher"
		class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
		<property name="jobRepository" ref="jobRepository" />
	</bean>

	<bean id="jobRepository"
		class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
		<property name="transactionManager" ref="transactionManager" />
	</bean>
	
</beans>

    5) database.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:jdbc="http://www.springframework.org/schema/jdbc" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.springframework.org/schema/beans 
		http://www.springframework.org/schema/beans/spring-beans-3.2.xsd
		http://www.springframework.org/schema/jdbc
		http://www.springframework.org/schema/jdbc/spring-jdbc-3.2.xsd">

	<!-- connect to database -->
	<bean id="derbyDataSource"
		class="org.springframework.jdbc.datasource.DriverManagerDataSource">
		<property name="driverClassName" value="org.apache.derby.jdbc.EmbeddedDriver" />
		<property name="url"
			value="jdbc:derby:C:/Documents and Settings/******/MyDB" />
		<property name="username" value="******" />
		<property name="password" value="******" />
	</bean>
	<bean id="transactionManager"
		class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
</beans>

    6) spring-batch-job-derby.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:batch="http://www.springframework.org/schema/batch" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="
		http://www.springframework.org/schema/beans 
		http://www.springframework.org/schema/beans/spring-beans-3.2.xsd
		http://www.springframework.org/schema/batch
		http://www.springframework.org/schema/batch/spring-batch-2.2.xsd">

	<bean id="domain" class="edu.xmu.spring.batch.model.Domain" scope="prototype"/>

	<batch:job id="readMultiFileJob">
		<batch:step id="readAndWriteCsvFile">
			<batch:tasklet>
				<batch:chunk reader="multiResourceReader" writer="derbyItemWriter"
					processor="flatFileItemProcessor" commit-interval="10" />
			</batch:tasklet>
		</batch:step>
	</batch:job>

	<bean id="multiResourceReader"
		class="org.springframework.batch.item.file.MultiResourceItemReader">
		<property name="resources" value="file:csv/input/domain-*.csv" />
		<property name="delegate" ref="flatFileItemReader" />
	</bean>
	<bean id="flatFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader">
		<property name="lineMapper">
			<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
				<property name="lineTokenizer">
					<bean
						class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
						<property name="names" value="id, username, password" />
					</bean>
				</property>
				<property name="fieldSetMapper">
					<bean
						class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
						<property name="prototypeBeanName" value="domain" />
					</bean>
				</property>
			</bean>
		</property>
	</bean>

	<bean id="flatFileItemProcessor" class="edu.xmu.spring.batch.processor.CustomCSVItemProcessor" />

	<bean id="derbyItemWriter"
		class="org.springframework.batch.item.database.JdbcBatchItemWriter">
		<property name="dataSource" ref="derbyDataSource" />
		<property name="sql">
			<value>
            <![CDATA[        
            	INSERT INTO ROOT.DOMAIN(ID, USERNAME, PASSWORD, DATE_ADDED) 
				VALUES (:id, :username, :password, :dateAdded)
            ]]>
			</value>
		</property>
		<property name="itemSqlParameterSourceProvider">
			<bean
				class="org.springframework.batch.item.database.BeanPropertyItemSqlParameterSourceProvider" />
		</property>
	</bean>

</beans>

    7) CustomCSVItemProcessor.java

package edu.xmu.spring.batch.processor;

import java.util.Calendar;
import java.util.Date;

import org.springframework.batch.item.ItemProcessor;

import edu.xmu.spring.batch.model.Domain;

public class CustomCSVItemProcessor implements ItemProcessor<Domain, Domain> {

    @Override
    public Domain process(Domain item) throws Exception {
	Date date = Calendar.getInstance().getTime();
	item.setUsername(item.getUsername().replace(',', '-')
		.replaceAll(" ", ""));
	item.setPassword(item.getPassword().replace(',', '-'));
	item.setDateAdded(date);
	return item;
    }

}

   8) App.java

package edu.xmu.spring.batch;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.context.support.ClassPathXmlApplicationContext;

public class App {
    public static void main(String[] args) {

	ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext(
		new String[] { "spring/batch/config/context.xml" });

	JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher");
	Job job = (Job) context.getBean("readMultiFileJob");

	JobExecution execution;
	try {
	    execution = jobLauncher.run(job, new JobParameters());
	    System.out.println("Exit Status : " + execution.getStatus());
	} catch (Exception e) {
	    e.printStackTrace();
	}

	System.out.println("Done");

	context.close();
    }
}

 

Reference Links:

1) http://www.mkyong.com/spring-batch/spring-batch-example-csv-file-to-database/

 

 

分享到:
评论

相关推荐

    Spring Recipes: A Problem-Solution Approach, Second Edition

    * Spring enterprise: Spring Java EE integration, Spring Integration, Spring Batch, jBPM with Spring, Spring Remoting, messaging, transactions, scaling using Terracotta and GridGrain, and more. ...

    spring-batch-3.0.5.RELEASE-dist.zip

    在"spring-batch-3.0.5.RELEASE-dist.zip"这个压缩包里,包含了Spring Batch框架的核心组件和相关文档,允许开发者快速地集成和配置批处理作业。以下是一些关键的知识点: 1. **核心概念**: - **Job**:Spring ...

    spring-batch-admin-1.3.0.RELEASE

    《Spring Batch Admin 1.3.0.RELEASE:全面解析与应用指南》 Spring Batch Admin是基于Spring Batch的管理界面,它为批处理作业的监控、管理和调度提供了一个直观的Web界面。在本文中,我们将深入探讨Spring Batch ...

    Spring Boot 2 Recipes: A Problem-Solution Approach

    You'll also get solutions to common problems with persistence, integrating Spring Boot with batch processing, algorithmic programming via Spring Batch, and much more. Other recipes cover topics such ...

    The Definitive Guide to Spring Batch, 2nd Edition.epub

    You’ll see how a new class of use cases and platforms has evolved to have an impact on batch-processing. Data science and big data have become prominent in modern IT and the use of batch processing ...

    spring-batch-admin-ui:Spring Batch管理员是一个最初采用spring boot 2,spring security,oauth2,Spring data jpa作为基础框架,集成了quartz提供调度能力,集成了Spring Batch提供批处理能力的管理系统。展示以及常见批处理的配置以及运行能力

    初始化数据库:找到目录:/ spring-batch-admin-backend / src / main / db,里面有两个文件,一个是数据库创建脚本,一个是表结构+数据的脚本,先执行创建库的,如果想在已经存在的库里面运行程序,可以省略这一步...

    magic-batch:演示项目-Java 8-Spring Boot-Redis

    使用Java 8+构建,Spring Boot(2.0.3)Redis(3.0.4) 经过JUnit(4.12)和JMeter的测试 通过带有三个Spring配置文件(dev,qa,dk)的Spring Boot执行-假定mvn clean install dev Profile异步处理批处理并注销...

    spring batch in action

    Jointly developed by SpringSource and Accenture, Spring Batch fills this critical gap by providing a robust and convenient framework for writing batch applications that process large volumes of ...

    spring-5.2.9.RELEASE-dist.zip(spring-framework-5.2.9.RELEASE)

    这些JAR包涵盖了Spring的核心模块,如core-container、web、data、amqp、batch等,还包括了对其他开源框架如Hibernate、MyBatis的支持库。开发者在实际项目中会把这些库导入到类路径中,以使用Spring提供的各种服务...

    org.springframework.batch-2.0.0.RELEASE-with-dependencies

    - "org.springframework.batch-2.0.0.RELEASE-with-dependencies" 压缩包包含了 Spring Batch 2.0.0.RELEASE 版本及其所有依赖,方便开发者快速搭建环境。 - 依赖包括但不限于 Spring Framework、JDBC 驱动、JPA、...

    poc_spring_batch:概念证明 - Spring批次

    Spring Batch 是一个强大的Java批处理框架,用于处理大量数据处理任务。这个名为“poc_spring_batch”的项目是一个概念证明,展示了如何利用Spring Batch来构建一个批量应用程序,并进行升级测试。下面将详细介绍...

    最新版完整包 spring-5.3.9.RELEASE-dist.zip

    11. **Spring Batch**:对于批量处理和工作流任务,Spring Batch 提供了一个全面的解决方案,包括事务管理、错误处理和重试机制。 12. **Spring Integration**:Spring Integration 提供了连接不同系统和协议的能力...

    README.mdgfdgdf

    |:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:| | 25 | 28.57 | 28.83 | 28.68 | 28.96 | 28.74 | 28.92 | **29.23** | **29.16** | **29.17** | - Set...

    spring batch 源码

    Spring Batch 是一个强大的、全面的批处理框架,用于处理大量数据。它被广泛应用于企业级应用,特别是那些需要处理大量输入/输出数据的系统。在深入理解 Spring Batch 的源码之前,我们需要先了解一下 Spring Batch ...

    spring-batch-2.2.0.RELEASE-no-dependencies

    《Spring Batch 2.2.0.RELEASE:批量处理与数据转换的核心技术解析》 Spring Batch 是一个由 Spring 社区开发的开源框架,专为处理大批量数据而设计。在2.2.0.RELEASE版本中,它提供了一套强大、可扩展且企业级的...

    Spring Recipes A Problem-Solution Approach(english version pdf)

    最后,本书还涵盖了Spring的其他扩展,如Spring Batch(批处理)、Spring Integration(企业级集成)和Spring Boot,这些都是Spring生态系统的一部分,有助于构建复杂的企业级应用。 通过《Spring Recipes: A ...

    bank-spring-batch:具有多处理器的Spring Batch项目

    【Spring Batch简介】 Spring Batch是Spring框架的一个模块,专门用于处理批量处理任务。它提供了一套强大、可扩展且易用的工具,用于处理大量数据的读取、处理和写入操作。Spring Batch的设计目标是解决企业级批量...

Global site tag (gtag.js) - Google Analytics