Batch Processing with Spring Batch

Some work is not request/response at all: importing a million-row CSV of students, generating monthly invoices for every customer, recalculating course statistics every night, migrating data between systems. These jobs must process large volumes efficiently, survive failures part-way through, restart where they stopped, skip or retry bad records, and report exactly what happened.

Spring Batch is the standard framework for this in Java. This lesson covers jobs, steps and chunk-oriented processing, readers, processors and writers, fault tolerance with skip and retry, restartability with a JDBC job repository, launching jobs, and what changed in Spring Batch 6 with Spring Boot 4.

Core Concepts

A Job is a batch process made of one or more Steps. The most common step type is chunk-oriented: an ItemReader reads items one by one, an optional ItemProcessor transforms or filters each, and an ItemWriter writes them in chunks (for example 500 at a time) within one transaction. A Tasklet step runs a single task instead, such as deleting a temporary file. Each run is a JobInstance identified by its JobParameters, with one or more JobExecution attempts.

The Job Repository and Restartability

Spring Batch records progress in a JobRepository: which executions ran, their status, and how far each step got. With a persistent repository, a failed job restarted with the same parameters resumes from the last committed chunk instead of starting over. In Spring Boot 4, spring-boot-starter-batch provides an in-memory (resourceless) repository — fine for simple, rerunnable jobs; add spring-boot-starter-batch-jdbc for a database-backed repository with restartability and history (MongoDB is supported too).

Readers and Writers

Spring Batch ships readers and writers for common sources: FlatFileItemReader/Writer for CSV and fixed-width files, JdbcCursorItemReader and JdbcPagingItemReader, JpaPagingItemReader, JdbcBatchItemWriter, JsonItemReader, Kafka and MongoDB readers/writers, and many more. Builders make configuration concise.

Fault Tolerance and Scaling

Mark a step faultTolerant() to skip bad records (up to a limit, logged via a SkipListener) and retry transient failures such as deadlocks. For large volumes, scale with multi-threaded steps, partitioning (split data into ranges processed in parallel), or remote chunking across machines.

Spring Batch 6 Changes

Spring Batch 6 reorganised packages (for example org.springframework.batch.core.job and ...core.step), made JobOperator the main API for launching jobs (it now extends JobLauncher), redesigned chunk-oriented steps — use .chunk(size).transactionManager(tx) instead of the deprecated .chunk(size, tx) — and added @EnableJdbcJobRepository/@EnableMongoJobRepository for store-specific configuration.

Examples

Dependencies and configuration

Java

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-batch-jdbc</artifactId>   <!-- persistent, restartable job repository -->
</dependency>
<dependency>
    <groupId>org.postgresql</groupId>
    <artifactId>postgresql</artifactId>
    <scope>runtime</scope>
</dependency>

# application.yml
spring:
  batch:
    jdbc:
      initialize-schema: always     # create BATCH_* metadata tables (use migrations in production)
    job:
      enabled: false                # don't run jobs automatically at startup; launch explicitly

Output

Created tables BATCH_JOB_INSTANCE, BATCH_JOB_EXECUTION, BATCH_STEP_EXECUTION, ...

A chunk-oriented job: import students from CSV into the database

Java

public record StudentCsv(String name, String email, String course) {}
public record StudentRow(String name, String email, String course) {}

@Configuration
public class ImportStudentsJobConfig {

    @Bean
    @StepScope
    FlatFileItemReader<StudentCsv> studentReader(@Value("#{jobParameters['file']}") String file) {
        return new FlatFileItemReaderBuilder<StudentCsv>()
            .name("studentReader")
            .resource(new FileSystemResource(file))
            .linesToSkip(1)                                    // header row
            .delimited()
            .names("name", "email", "course")
            .targetType(StudentCsv.class)
            .build();
    }

    @Bean
    ItemProcessor<StudentCsv, StudentRow> studentProcessor() {
        return csv -> {
            if (csv.email() == null || !csv.email().contains("@")) {
                throw new ValidationException("Invalid email for " + csv.name());   // will be skipped
            }
            return new StudentRow(csv.name().trim(), csv.email().toLowerCase().trim(), csv.course());
        };
    }

    @Bean
    JdbcBatchItemWriter<StudentRow> studentWriter(DataSource dataSource) {
        return new JdbcBatchItemWriterBuilder<StudentRow>()
            .dataSource(dataSource)
            .sql("insert into student (name, email, course) values (:name, :email, :course) "
               + "on conflict (email) do nothing")
            .beanMapped()
            .build();
    }

    @Bean
    Step importStep(JobRepository jobRepository, PlatformTransactionManager tx,
                    FlatFileItemReader<StudentCsv> reader,
                    ItemProcessor<StudentCsv, StudentRow> processor,
                    JdbcBatchItemWriter<StudentRow> writer) {
        return new StepBuilder("importStudents", jobRepository)
            .<StudentCsv, StudentRow>chunk(500)
            .transactionManager(tx)
            .reader(reader)
            .processor(processor)
            .writer(writer)
            .faultTolerant()
            .skip(ValidationException.class)
            .skipLimit(100)
            .retry(DeadlockLoserDataAccessException.class)
            .retryLimit(3)
            .build();
    }

    @Bean
    Job importStudentsJob(JobRepository jobRepository, Step importStep) {
        return new JobBuilder("importStudentsJob", jobRepository)
            .start(importStep)
            .build();
    }
}

Output

(Reads 500 rows, processes them, writes them in one transaction, commits, and records progress — repeated until the file ends.)

Launching the job with JobOperator and inspecting the result

Java

@RestController
@RequestMapping("/admin/jobs")
public class JobController {

    private final JobOperator jobOperator;
    private final Job importStudentsJob;

    public JobController(JobOperator jobOperator, Job importStudentsJob) {
        this.jobOperator = jobOperator;
        this.importStudentsJob = importStudentsJob;
    }

    @PostMapping("/import-students")
    public String run(@RequestParam String file) throws Exception {
        JobParameters params = new JobParametersBuilder()
            .addString("file", file)
            .addLocalDateTime("requestedAt", LocalDateTime.now())   // makes each run a new instance
            .toJobParameters();
        JobExecution execution = jobOperator.start(importStudentsJob, params);
        return execution.getStatus() + " " + execution.getStepExecutions().stream()
            .map(s -> s.getStepName() + ": read=" + s.getReadCount() + " written=" + s.getWriteCount()
                      + " skipped=" + s.getSkipCount())
            .toList();
    }
}

Output

POST /admin/jobs/import-students?file=D:/imports/students-2026-09.csv
COMPLETED [importStudents: read=120000 written=119986 skipped=14]

(crash at row 64,000 and restart with the same file parameter)
-> resumes from row 64,001 because chunks up to 64,000 were already committed

Common Mistakes

Loading an entire file or table into memory in a custom loop instead of using chunk-oriented processing.
Using only the in-memory job repository for long jobs that must be restartable.
Leaving spring.batch.job.enabled at its default in web applications, so jobs run on every startup.
Reusing identical JobParameters for a new run and getting JobInstanceAlreadyCompleteException.
Using the deprecated chunk(size, transactionManager) and JobLauncher APIs in new Spring Batch 6 code.

Key Points to Remember

A Job contains Steps; chunk steps read, process and write items in transactional chunks.
spring-boot-starter-batch-jdbc gives a persistent JobRepository for history and restarts.
Built-in readers/writers cover files, JDBC, JPA, JSON, Kafka and more.
faultTolerant() adds skip and retry policies; partitioning and multi-threading scale out.
Spring Batch 6: JobOperator to launch, chunk(n).transactionManager(tx), new packages.

Practice the examples

Change an input, predict the result, then compare it with the output. Explain why the result changes.

Use your local JDK or project IDE for these examples. Codelab currently runs Python and HTML/CSS/JavaScript; framework examples may need project dependencies.