Programming Cookbook Blog

In this post of Spring Batch, I will explain how to write the records read, to multiple files instead of one.

We take the help of MultiResourceItemWriter, it writes to multiple file and suffixes the output fileNames with an index.

For example if the file name to be created, is DemoData.txt and two files are created, the name of two files will be DemoData.txt.1 and DemoData.txt.2.

The MultiResourceItemWriter takes three arguments
1) The file name including the path
2) The writer bean which will do the actual writing
3) A count indicating the number of records to be written in one file.

Below is the xml configuration

1   <bean id="writer" class="org.springframework.batch.item.file.FlatFileItemWriter">
2       <property name="lineAggregator">
3           <bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
4               <property name="fieldExtractor">
5                   <bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
6                       <property name="names" value="id,name,status,salary"/>
7                   </bean>
8               </property>
9           </bean>
10      </property>
11  </bean>
12  
13  <bean id="multiFileWriter" class="org.springframework.batch.item.file.MultiResourceItemWriter">
14      <property name="resource" value="file:DemoData.txt"/>
15      <property name="itemCountLimitPerResource" value="50"/>
16      <property name="delegate" ref="writer"/>
17  </bean>

From line 1 to 11, we define the bean writer, the definition of the bean is same as the bean definitions in previous posts.

From line 13 to 17, we define a new bean of type MultiResourceItemWriter, and the name of the bean is multiFileWriter.

It takes the file name to be created as DemoData.txt. The itemCountLimitPerResource indicates max 50 records can be present in the file. It also refer to writer bean by
setting the delegate property.

Now we integrate into the job as shown in the below xml configuration

1   <batch:job id="importProductsJob">
2       <batch:step id="readWriteProducts">
3           <batch:tasklet>
4               <batch:chunk reader="reader" writer="multiFileWriter" commit-interval="50"/>
5           </batch:tasklet>
6       </batch:step>
7   </batch:job>

At line 4, we set the property “writer” to the bean multiFileWriter instead of to the actual writer bean.

If the source file has 100 records and itemCountLimitPerResource is 50. Then two files are created with name DemoData.txt.1 and DemoData.txt.2, each file having 50 records.

Below is the complete xml configuration file

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:batch="http://www.springframework.org/schema/batch"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
                        http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch.xsd">
    
    <bean id="employee" class="package7.Employee" scope="prototype"/>
    
    <bean id="reader" class="org.springframework.batch.item.file.FlatFileItemReader">
        <property name="resource" value="file:FileInput.txt"/>
        <property name="lineMapper">
            <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
                <property name="lineTokenizer">
                    <bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
                        <property name="names" value="id,name,status,salary"/>
                    </bean>
                </property>
                <property name="fieldSetMapper">
                    <bean class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
                        <property name="prototypeBeanName" value="employee"/>
                    </bean>
                </property>
            </bean>
        </property> 
    </bean>
    
    <bean id="writer" class="org.springframework.batch.item.file.FlatFileItemWriter">
        <property name="lineAggregator">
            <bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
                <property name="fieldExtractor">
                    <bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
                        <property name="names" value="id,name,status,salary"/>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>
    
    <bean id="multiFileWriter" class="org.springframework.batch.item.file.MultiResourceItemWriter">
        <property name="resource" value="file:DemoData.txt"/>
        <property name="itemCountLimitPerResource" value="50"/>
        <property name="delegate" ref="writer"/>
    </bean>
    
    <batch:job id="importProductsJob">
        <batch:step id="readWriteProducts">
            <batch:tasklet>
                <batch:chunk reader="reader" writer="multiFileWriter" commit-interval="50"/>
            </batch:tasklet>
        </batch:step>
    </batch:job>
    
    <bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
    
    <bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
        <property name="transactionManager" ref="transactionManager"/>
    </bean>
    
    <bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
        <property name="jobRepository" ref="jobRepository"/>
    </bean>
</beans>