By WebNest Studio Engineering Team
Spring Boot Tutorial
Running Local Models with Ollama
Hosted models are powerful, but they are not always an option: some data may not leave your network, API costs add up during development, and you may want to work offline. Ollama runs open-weight models — Llama, Mistral, Gemma, Qwen, DeepSeek, Phi and many more — locally on your laptop or on your own servers, with a simple HTTP API.
Spring AI supports Ollama for both chat and embeddings, and because your code uses the portable ChatClient and EmbeddingModel APIs, you can develop against a local model and switch to a hosted one in production (or the reverse) with configuration alone. This lesson covers installing Ollama, connecting Spring Boot to it, running it with Docker Compose, and using several providers in one application.
Installing Ollama and Pulling Models
Download Ollama from ollama.com for Windows, macOS or Linux, or run the official ollama/ollama Docker image. Pull a model with ollama pull llama3.2 and an embedding model with ollama pull nomic-embed-text. Ollama serves its API on http://localhost:11434.
Choose model size to fit your hardware: 1–4B parameter models run on most laptops; 7–8B models want 8–16 GB of RAM or a GPU; larger models need serious GPUs. Smaller models are faster but weaker at reasoning, tool calling and structured output.
Configuring Spring AI for Ollama
Add spring-ai-starter-model-ollama, then set spring.ai.ollama.base-url, spring.ai.ollama.chat.model and spring.ai.ollama.embedding.model. Spring AI can pull missing models automatically at startup with spring.ai.ollama.init.pull-model-strategy=when_missing, which is convenient for development and CI.
Docker Compose and Testcontainers
Spring Boot's Docker Compose support detects an ollama/ollama service in compose.yaml, starts it with the application, and configures the base URL automatically through a service connection. Testcontainers offers an OllamaContainer for integration tests, also connected with @ServiceConnection.
Using Several Model Providers
With more than one model starter on the classpath, the auto-configured ChatClient.Builder becomes ambiguous. Disable it with spring.ai.chat.client.enabled=false and build a ChatClient for each ChatModel bean yourself — for example a local Ollama model for cheap classification and a hosted model for complex reasoning.
Examples
Installing Ollama and pulling models
# After installing from ollama.com
ollama pull llama3.2
ollama pull nomic-embed-text
ollama list
# Quick test from the terminal
ollama run llama3.2 "Explain Spring Boot starters in one sentence."
NAME SIZE
llama3.2:latest 2.0 GB
nomic-embed-text:latest 274 MB
Spring Boot starters are dependency bundles that pull in everything needed for a feature, like web or JPA, with compatible versions.
Spring Boot configuration for Ollama
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-starter-model-ollama</artifactId>
</dependency>
# application.yml
spring:
ai:
ollama:
base-url: http://localhost:11434
init:
pull-model-strategy: when_missing
chat:
model: llama3.2
temperature: 0.2
embedding:
model: nomic-embed-text
INFO Pulling model llama3.2 (strategy when_missing)... already available
INFO Started WebnestAiApplication in 2.1 seconds
Running Ollama with Docker Compose
# compose.yaml in the project root
services:
ollama:
image: ollama/ollama:latest
ports:
- "11434:11434"
volumes:
- ollama-data:/root/.ollama
volumes:
ollama-data:
<!-- pom.xml: Spring Boot starts compose.yaml automatically in development -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-docker-compose</artifactId>
<optional>true</optional>
</dependency>
INFO Using Docker Compose file compose.yaml
INFO Container project-ollama-1 Started
(spring.ai.ollama.base-url is set automatically from the running container)
Local model for classification, hosted model for complex answers
# application.yml
spring:
ai:
chat:
client:
enabled: false # we create ChatClients ourselves
@Configuration
public class MultiModelConfig {
@Bean
ChatClient localClient(OllamaChatModel ollama) {
return ChatClient.builder(ollama)
.defaultSystem("Classify text. Reply with a single word.")
.build();
}
@Bean
ChatClient cloudClient(OpenAiChatModel openAi) {
return ChatClient.builder(openAi)
.defaultSystem("You are an expert Spring Boot architect.")
.build();
}
}
@Service
public class QuestionRouter {
private final ChatClient localClient;
private final ChatClient cloudClient;
public QuestionRouter(ChatClient localClient, ChatClient cloudClient) {
this.localClient = localClient;
this.cloudClient = cloudClient;
}
public String answer(String question) {
String kind = localClient.prompt()
.user("Is this question SIMPLE or COMPLEX? " + question)
.call().content().trim();
return kind.startsWith("COMPLEX")
? cloudClient.prompt().user(question).call().content()
: localClient.prompt().system("Answer briefly.").user(question).call().content();
}
}
answer("What port does Spring Boot use by default?") -> (local) 8080.
answer("Design a multi-tenant schema strategy for a SaaS on Spring Boot") -> (cloud) There are three common strategies...
Common Mistakes
- Expecting a small local model to match a frontier hosted model at tool calling or complex reasoning — test each feature with the model you will deploy.
- Forgetting to pull the model, then seeing "model not found" errors; use pull-model-strategy or pull it in your setup script.
- Running Ollama in Docker on a machine with a GPU without enabling GPU access, making it far slower than necessary.
- Adding both OpenAI and Ollama starters and getting "expected single matching bean" errors for ChatClient.Builder.
- Mixing embedding models: documents embedded with nomic-embed-text cannot be searched with OpenAI embeddings.
Exercise
A teammate is about to make this mistake: Expecting a small local model to match a frontier hosted model at tool calling or complex reasoning — test each feature with the model you will deploy. Use Installing Ollama and Pulling Models to review the decision. Write a corrected implementation or design sketch that follows Ollama runs open-weight models locally with an HTTP API on port 11434., then name one edge case you would check.
Show hints
- Start with Installing Ollama and Pulling Models and compare your approach with Installing Ollama and pulling models. Identify the decision or behavior the lesson recommends.
- Walk through your scenario step by step. Confirm that it follows Ollama runs open-weight models locally with an HTTP API on port 11434. and that it would catch the failure described by Expecting a small local model to match a frontier hosted model at tool calling or complex reasoning — test each feature with the model you will deploy..
Quick quiz
1. Which choice best describes the recommended approach in Running Local Models with Ollama?
- A. Expecting a small local model to match a frontier hosted model at tool calling or complex reasoning — test each feature with the model you will deploy.
- B. Forgetting to pull the model, then seeing "model not found" errors; use pull-model-strategy or pull it in your setup script.
- C. Ollama runs open-weight models locally with an HTTP API on port 11434.
Show answer
C. This is the recommended approach: Ollama runs open-weight models locally with an HTTP API on port 11434. Avoid: Expecting a small local model to match a frontier hosted model at tool calling or complex reasoning — test each feature with the model you will deploy.
Interview questions
Why does this principle matter in Running Local Models with Ollama: Ollama runs open-weight models locally with an HTTP API on port 11434.
Show answer
Ollama runs open-weight models locally with an HTTP API on port 11434. Apply this in Installing Ollama and Pulling Models; the example Installing Ollama and pulling models shows one way to do it. Avoid Expecting a small local model to match a frontier hosted model at tool calling or complex reasoning — test each feature with the model you will deploy. and explain how your implementation or design prevents that failure.
Key Points to Remember
- Ollama runs open-weight models locally with an HTTP API on port 11434.
- spring-ai-starter-model-ollama plus spring.ai.ollama.* properties connect Spring AI to it.
- Docker Compose support and Testcontainers wire up Ollama automatically through service connections.
- Portable ChatClient code lets you switch between local and hosted models via configuration.
- With several providers, disable the auto-configured builder and create one ChatClient per ChatModel.
Practice in the playground
Write and run your own code for this topic, or start from the exercise or an example. Runs plain Java 17. Frameworks such as Spring, and databases, are not available here. Your code for each topic is kept in this browser.
Run the examples
Change an input, predict the result, then compare it with the output. Explain why the result changes.
These examples need a JDK project with their libraries on your computer, so they can be copied but not run here. Codelab runs Python, FastAPI backends with SQLite, plain Java 17 and HTML/CSS/JavaScript.
Written by
WebNest Studio Engineering Team
Practical lessons from developers working with Java, Python, React, databases and production systems.
Led by Vansh Duggal, Founder, WebNest Studio · Full-stack & software engineering