Skipping records during record reading, processing or writing

While reading a record from a file, processing already read record, or writing a record to a file, exceptions happens.

They are two ways to handle the exceptions
1) skip it and continue with the next record.
2) Stop the processing of entire file.

Sometimes the exceptions are not so serious and we can skip it.

In this post under Spring Batch, I will explain how to implement the first approach.

We can configure the Spring Batch to skip the current record if an exception happens. As shown below

XML code


1   <batch:job id="importEmployees">
2       <batch:step id="readWriteEmployees">
3           <batch:tasklet>
4               <batch:chunk reader="reader" writer="writer" commit-interval="50" skip-limit="200">
5                   <batch:skippable-exception-classes>
6                       <batch:include class="org.springframework.batch.item.file.FlatFileParseException"/>
7                   </batch:skippable-exception-classes>
8               </batch:chunk>
9           </batch:tasklet>
10      </batch:step>
11  </batch:job>

In the above xml code and line number 4, we have added “skip-limit” attribute which takes an integer value. The integer value represent the number of records (that throw exceptions) which can be skipped.

In our example the value is 200. So if more than 200 records throw an exception, the job execution is stopped.

Next we have to tell the job framework, for which type of exception the record can be skipped. For that to configure we are using “skippable-exception-classes” element as shown at line 5.

The exception for which the record can be skipped is added using the “include” element as a child of “skippable-exception-classes” element as shown at line 6. In our example the exception is FlatFileParseException.

In this way we configure two things
1) the number of exceptions that are allowed to be skipped
2) the type of exception that is allowed to be skipped

Below is the complete xml for your reference

job.xml


<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:batch="http://www.springframework.org/schema/batch"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
                        http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch.xsd">
    
    <bean id="employee" class="package24.Employee" scope="prototype"/>
    
    <bean id="reader" class="org.springframework.batch.item.file.FlatFileItemReader">
        <property name="resource" value="file:FileInput8.txt"/>
        <property name="lineMapper">
            <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
                <property name="lineTokenizer">
                    <bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
                        <property name="names" value="id,name,status,salary"/>
                    </bean>
                </property>
                <property name="fieldSetMapper">
                    <bean class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
                        <property name="prototypeBeanName" value="employee"/>
                    </bean>
                </property>
            </bean>
        </property> 
    </bean>
    
    <bean id="writer" class="org.springframework.batch.item.file.FlatFileItemWriter">
        <property name="resource" value="file:FileOutput.txt"/>
        <property name="lineAggregator">
            <bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
                <property name="fieldExtractor">
                    <bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
                        <property name="names" value="id,name,status,salary"/>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>
    
    <batch:job id="importEmployees">
        <batch:step id="readWriteEmployees">
            <batch:tasklet>
                <batch:chunk reader="reader" writer="writer" commit-interval="50" skip-limit="200">
                    <batch:skippable-exception-classes>
                        <batch:include class="org.springframework.batch.item.file.FlatFileParseException"/>
                    </batch:skippable-exception-classes>
                </batch:chunk>
            </batch:tasklet>
        </batch:step>
    </batch:job>
    
    <bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
    
    <bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
        <property name="transactionManager" ref="transactionManager"/>
    </bean>
    
    <bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
        <property name="jobRepository" ref="jobRepository"/>
    </bean>
</beans>

Leave a Reply