Metrics with Micrometer and Prometheus

"Is the application healthy?" is not a yes/no question in production. You want to know how many requests per second it serves, how long they take at the 99th percentile, how many fail, how full the connection pool is, and how many orders were placed in the last hour. Metrics answer these questions continuously, and they drive dashboards and alerts.

Spring Boot uses Micrometer, a vendor-neutral metrics facade (think SLF4J for metrics), and exports to Prometheus, Datadog, New Relic, CloudWatch, OTLP and many others. This lesson covers built-in metrics, exposing them to Prometheus, creating custom counters, timers and gauges, the @Timed and @Observed annotations, tags and cardinality, and building Grafana dashboards and alerts.

Built-in Metrics

With Actuator on the classpath, Spring Boot records metrics automatically: HTTP server requests (http.server.requests with uri, method, status tags), HTTP client calls, JVM memory and GC, threads, CPU, HikariCP pool usage, Tomcat sessions, cache hit ratios, Spring Data repository calls, Kafka and RabbitMQ consumers, scheduled tasks, and application startup time. You get useful dashboards before writing any metric code.

Exporting to Prometheus

Add micrometer-registry-prometheus and expose the prometheus Actuator endpoint. Prometheus scrapes /actuator/prometheus every few seconds and stores time series; Grafana visualises them. On Kubernetes, the Prometheus Operator discovers pods through annotations or ServiceMonitors. Alternatively, push metrics via OTLP to an OpenTelemetry collector.

Meter Types

Micrometer offers a small set of meter types:

Counter — a value that only increases: orders placed, emails sent, errors.
Timer — count and duration of events, with percentiles and histograms: request latency, payment processing time.
Gauge — a current value that goes up and down: queue size, active users, cache size.
DistributionSummary — distribution of non-time values: order amounts, payload sizes.
LongTaskTimer — currently running long tasks and their duration: batch jobs, report generation.

Tags and Cardinality

Tags (dimensions) let you slice metrics — by payment method, country, outcome. But every unique combination of tag values creates a separate time series. Never tag with unbounded values such as user id, order id, email or full URL with ids; that "cardinality explosion" can overwhelm Prometheus and your bill. Use a small, fixed set of values.

Observations

Micrometer's Observation API (@Observed or Observation.createNotStarted(...)) instruments a piece of code once and produces both a timer metric and a tracing span. Spring itself uses observations for HTTP, data access and messaging, which is why metrics and traces line up in Spring Boot.

Examples

Exposing metrics to Prometheus

Java

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>

# application.yml
management:
  endpoints:
    web:
      exposure:
        include: health, info, prometheus
  metrics:
    tags:
      application: ${spring.application.name}     # common tag on every metric
    distribution:
      percentiles-histogram:
        http.server.requests: true                # enables p95/p99 in Prometheus

Output

curl localhost:8080/actuator/prometheus | findstr http_server_requests
http_server_requests_seconds_count{application="webnest-shop",method="GET",outcome="SUCCESS",status="200",uri="/api/courses/{slug}"} 1842
http_server_requests_seconds_sum{...,uri="/api/courses/{slug}"} 41.27
hikaricp_connections_active{application="webnest-shop",pool="HikariPool-1"} 3

Custom counters, timers, gauges and distribution summaries

Java

@Service
public class CheckoutMetrics {

    private final Counter ordersPlaced;
    private final Timer paymentTimer;
    private final DistributionSummary orderAmount;
    private final MeterRegistry registry;

    public CheckoutMetrics(MeterRegistry registry, PendingOrderQueue queue) {
        this.registry = registry;
        this.ordersPlaced = Counter.builder("shop.orders.placed")
            .description("Orders successfully placed")
            .register(registry);
        this.paymentTimer = Timer.builder("shop.payment.duration")
            .publishPercentiles(0.5, 0.95, 0.99)
            .register(registry);
        this.orderAmount = DistributionSummary.builder("shop.order.amount")
            .baseUnit("INR")
            .register(registry);
        Gauge.builder("shop.orders.pending", queue, PendingOrderQueue::size)
            .register(registry);
    }

    public void orderPlaced(BigDecimal amount, String paymentMethod) {
        ordersPlaced.increment();
        orderAmount.record(amount.doubleValue());
        // tag with a bounded set of values only
        registry.counter("shop.orders.by_method", "method", paymentMethod).increment();
    }

    public <T> T timePayment(Supplier<T> payment) {
        return paymentTimer.record(payment);
    }
}

Output

shop_orders_placed_total{application="webnest-shop"} 412.0
shop_orders_by_method_total{method="upi"} 280.0
shop_orders_by_method_total{method="card"} 132.0
shop_payment_duration_seconds{quantile="0.99"} 1.84
shop_orders_pending 7.0

@Observed for a metric and a trace span in one annotation

Java

@Configuration
public class ObservationConfig {
    @Bean
    ObservedAspect observedAspect(ObservationRegistry registry) {   // needs spring-boot-starter-aspectj
        return new ObservedAspect(registry);
    }
}

@Service
public class RecommendationService {

    @Observed(name = "recommendations.compute",
              contextualName = "compute-recommendations",
              lowCardinalityKeyValues = {"algorithm", "collaborative"})
    public List<Course> recommend(long studentId) {
        // ...
        return List.of();
    }
}

Output

recommendations_compute_seconds_count{algorithm="collaborative",error="none"} 96
(and a span named "compute-recommendations" appears in the trace of each request)

Grafana queries and a Prometheus alert rule

Java

# p99 latency per endpoint over 5 minutes (PromQL)
histogram_quantile(0.99, sum by (le, uri) (rate(http_server_requests_seconds_bucket{application="webnest-shop"}[5m])))

# error rate
sum(rate(http_server_requests_seconds_count{status=~"5.."}[5m]))
  / sum(rate(http_server_requests_seconds_count[5m]))

# alert.rules.yml
groups:
  - name: webnest-shop
    rules:
      - alert: HighErrorRate
        expr: |
          sum(rate(http_server_requests_seconds_count{application="webnest-shop",status=~"5.."}[5m]))
            / sum(rate(http_server_requests_seconds_count{application="webnest-shop"}[5m])) > 0.05
        for: 10m
        labels:
          severity: page
        annotations:
          summary: "More than 5% of requests are failing"

Output

Grafana panel: p99 /api/checkout = 820 ms, /api/courses/{slug} = 45 ms
Alert HighErrorRate FIRING after 10 minutes above 5%

Common Mistakes

Tagging metrics with user ids, order ids or raw URLs, creating millions of time series.
Exposing /actuator/prometheus publicly instead of on an internal port or network.
Measuring only averages; latency problems show up in p95/p99 percentiles.
Creating new meters on every call with different names instead of registering them once and reusing them.
Collecting metrics but never defining alerts, so problems are only noticed by users.

Key Points to Remember

Micrometer is the metrics facade; Spring Boot records HTTP, JVM, pool, cache and messaging metrics automatically.
micrometer-registry-prometheus + the prometheus endpoint lets Prometheus scrape metrics.
Use Counter, Timer, Gauge, DistributionSummary and LongTaskTimer for business metrics.
Keep tag values bounded to avoid cardinality explosions.
@Observed produces both metrics and tracing spans; build dashboards and alerts on top.

Practice the examples

Change an input, predict the result, then compare it with the output. Explain why the result changes.

Use your local JDK or project IDE for these examples. Codelab currently runs Python and HTML/CSS/JavaScript; framework examples may need project dependencies.