Course topics

By WebNest Studio Engineering Team

Spring Boot Tutorial

Batch Processing with Spring Batch

Some work is not request/response at all: importing a million-row CSV of students, generating monthly invoices for every customer, recalculating course statistics every night, migrating data between systems. These jobs must process large volumes efficiently, survive failures part-way through, restart where they stopped, skip or retry bad records, and report exactly what happened.

Spring Batch is the standard framework for this in Java. This lesson covers jobs, steps and chunk-oriented processing, readers, processors and writers, fault tolerance with skip and retry, restartability with a JDBC job repository, launching jobs, and what changed in Spring Batch 6 with Spring Boot 4.

Core Concepts

A Job is a batch process made of one or more Steps. The most common step type is chunk-oriented: an ItemReader reads items one by one, an optional ItemProcessor transforms or filters each, and an ItemWriter writes them in chunks (for example 500 at a time) within one transaction. A Tasklet step runs a single task instead, such as deleting a temporary file. Each run is a JobInstance identified by its JobParameters, with one or more JobExecution attempts.

The Job Repository and Restartability

Spring Batch records progress in a JobRepository: which executions ran, their status, and how far each step got. With a persistent repository, a failed job restarted with the same parameters resumes from the last committed chunk instead of starting over. In Spring Boot 4, spring-boot-starter-batch provides an in-memory (resourceless) repository — fine for simple, rerunnable jobs; add spring-boot-starter-batch-jdbc for a database-backed repository with restartability and history (MongoDB is supported too).

Readers and Writers

Spring Batch ships readers and writers for common sources: FlatFileItemReader/Writer for CSV and fixed-width files, JdbcCursorItemReader and JdbcPagingItemReader, JpaPagingItemReader, JdbcBatchItemWriter, JsonItemReader, Kafka and MongoDB readers/writers, and many more. Builders make configuration concise.

Fault Tolerance and Scaling

Mark a step faultTolerant() to skip bad records (up to a limit, logged via a SkipListener) and retry transient failures such as deadlocks. For large volumes, scale with multi-threaded steps, partitioning (split data into ranges processed in parallel), or remote chunking across machines.

Spring Batch 6 Changes

Spring Batch 6 reorganised packages (for example org.springframework.batch.core.job and ...core.step), made JobOperator the main API for launching jobs (it now extends JobLauncher), redesigned chunk-oriented steps — use .chunk(size).transactionManager(tx) instead of the deprecated .chunk(size, tx) — and added @EnableJdbcJobRepository/@EnableMongoJobRepository for store-specific configuration.

Examples

Dependencies and configuration

Java
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-batch-jdbc</artifactId>   <!-- persistent, restartable job repository -->
</dependency>
<dependency>
    <groupId>org.postgresql</groupId>
    <artifactId>postgresql</artifactId>
    <scope>runtime</scope>
</dependency>

# application.yml
spring:
  batch:
    jdbc:
      initialize-schema: always     # create BATCH_* metadata tables (use migrations in production)
    job:
      enabled: false                # don't run jobs automatically at startup; launch explicitly
Output
Created tables BATCH_JOB_INSTANCE, BATCH_JOB_EXECUTION, BATCH_STEP_EXECUTION, ...

A chunk-oriented job: import students from CSV into the database

Java
public record StudentCsv(String name, String email, String course) {}
public record StudentRow(String name, String email, String course) {}

@Configuration
public class ImportStudentsJobConfig {

    @Bean
    @StepScope
    FlatFileItemReader<StudentCsv> studentReader(@Value("#{jobParameters['file']}") String file) {
        return new FlatFileItemReaderBuilder<StudentCsv>()
            .name("studentReader")
            .resource(new FileSystemResource(file))
            .linesToSkip(1)                                    // header row
            .delimited()
            .names("name", "email", "course")
            .targetType(StudentCsv.class)
            .build();
    }

    @Bean
    ItemProcessor<StudentCsv, StudentRow> studentProcessor() {
        return csv -> {
            if (csv.email() == null || !csv.email().contains("@")) {
                throw new ValidationException("Invalid email for " + csv.name());   // will be skipped
            }
            return new StudentRow(csv.name().trim(), csv.email().toLowerCase().trim(), csv.course());
        };
    }

    @Bean
    JdbcBatchItemWriter<StudentRow> studentWriter(DataSource dataSource) {
        return new JdbcBatchItemWriterBuilder<StudentRow>()
            .dataSource(dataSource)
            .sql("insert into student (name, email, course) values (:name, :email, :course) "
               + "on conflict (email) do nothing")
            .beanMapped()
            .build();
    }

    @Bean
    Step importStep(JobRepository jobRepository, PlatformTransactionManager tx,
                    FlatFileItemReader<StudentCsv> reader,
                    ItemProcessor<StudentCsv, StudentRow> processor,
                    JdbcBatchItemWriter<StudentRow> writer) {
        return new StepBuilder("importStudents", jobRepository)
            .<StudentCsv, StudentRow>chunk(500)
            .transactionManager(tx)
            .reader(reader)
            .processor(processor)
            .writer(writer)
            .faultTolerant()
            .skip(ValidationException.class)
            .skipLimit(100)
            .retry(DeadlockLoserDataAccessException.class)
            .retryLimit(3)
            .build();
    }

    @Bean
    Job importStudentsJob(JobRepository jobRepository, Step importStep) {
        return new JobBuilder("importStudentsJob", jobRepository)
            .start(importStep)
            .build();
    }
}
Output
(Reads 500 rows, processes them, writes them in one transaction, commits, and records progress — repeated until the file ends.)

Launching the job with JobOperator and inspecting the result

Java
@RestController
@RequestMapping("/admin/jobs")
public class JobController {

    private final JobOperator jobOperator;
    private final Job importStudentsJob;

    public JobController(JobOperator jobOperator, Job importStudentsJob) {
        this.jobOperator = jobOperator;
        this.importStudentsJob = importStudentsJob;
    }

    @PostMapping("/import-students")
    public String run(@RequestParam String file) throws Exception {
        JobParameters params = new JobParametersBuilder()
            .addString("file", file)
            .addLocalDateTime("requestedAt", LocalDateTime.now())   // makes each run a new instance
            .toJobParameters();
        JobExecution execution = jobOperator.start(importStudentsJob, params);
        return execution.getStatus() + " " + execution.getStepExecutions().stream()
            .map(s -> s.getStepName() + ": read=" + s.getReadCount() + " written=" + s.getWriteCount()
                      + " skipped=" + s.getSkipCount())
            .toList();
    }
}
Output
POST /admin/jobs/import-students?file=D:/imports/students-2026-09.csv
COMPLETED [importStudents: read=120000 written=119986 skipped=14]

(crash at row 64,000 and restart with the same file parameter)
-> resumes from row 64,001 because chunks up to 64,000 were already committed

Common Mistakes

  • Loading an entire file or table into memory in a custom loop instead of using chunk-oriented processing.
  • Using only the in-memory job repository for long jobs that must be restartable.
  • Leaving spring.batch.job.enabled at its default in web applications, so jobs run on every startup.
  • Reusing identical JobParameters for a new run and getting JobInstanceAlreadyCompleteException.
  • Using the deprecated chunk(size, transactionManager) and JobLauncher APIs in new Spring Batch 6 code.

Exercise

Using the lesson's Dependencies and configuration example, create a small implementation or design sketch that demonstrates A Job contains Steps; chunk steps read, process and write items in transactional chunks.. Add a test case or scenario that would catch Loading an entire file or table into memory in a custom loop instead of using chunk-oriented processing., and describe the expected result.

Show hints
  • Start with Core Concepts and compare your approach with Dependencies and configuration. Identify the decision or behavior the lesson recommends.
  • Walk through your scenario step by step. Confirm that it follows A Job contains Steps; chunk steps read, process and write items in transactional chunks. and that it would catch the failure described by Loading an entire file or table into memory in a custom loop instead of using chunk-oriented processing..

Quick quiz

1. Which choice is a mistake to avoid in Batch Processing with Spring Batch?

  • A. A Job contains Steps; chunk steps read, process and write items in transactional chunks.
  • B. Loading an entire file or table into memory in a custom loop instead of using chunk-oriented processing.
  • C. Using only the in-memory job repository for long jobs that must be restartable.
Show answer

B. Avoid this approach: Loading an entire file or table into memory in a custom loop instead of using chunk-oriented processing. The lesson recommends: A Job contains Steps; chunk steps read, process and write items in transactional chunks.

Interview questions

How would you apply the guidance in Core Concepts to a real task, and what failure would you guard against?

Show answer

A Job contains Steps; chunk steps read, process and write items in transactional chunks. Apply this in Core Concepts; the example Dependencies and configuration shows one way to do it. Avoid Loading an entire file or table into memory in a custom loop instead of using chunk-oriented processing. and explain how your implementation or design prevents that failure.

Key Points to Remember

  • A Job contains Steps; chunk steps read, process and write items in transactional chunks.
  • spring-boot-starter-batch-jdbc gives a persistent JobRepository for history and restarts.
  • Built-in readers/writers cover files, JDBC, JPA, JSON, Kafka and more.
  • faultTolerant() adds skip and retry policies; partitioning and multi-threading scale out.
  • Spring Batch 6: JobOperator to launch, chunk(n).transactionManager(tx), new packages.

Practice in the playground

Write and run your own code for this topic, or start from the exercise or an example. Runs plain Java 17. Frameworks such as Spring, and databases, are not available here. Your code for each topic is kept in this browser.

Run the examples

Change an input, predict the result, then compare it with the output. Explain why the result changes.

These examples need a JDK project with their libraries on your computer, so they can be copied but not run here. Codelab runs Python, FastAPI backends with SQLite, plain Java 17 and HTML/CSS/JavaScript.

WebNest Studio

Written by

WebNest Studio Engineering Team

Practical lessons from developers working with Java, Python, React, databases and production systems.

Led by Vansh Duggal, Founder, WebNest Studio · Full-stack & software engineering