Backend Study Lab

Production Observability & SRE · Lesson 7

JVM Profiling and JFR for Production Backends

Use Java 25 Flight Recorder to investigate CPU, allocation, GC, locks, I/O, and virtual threads without overstating profile evidence.

By CloveCode Studio Aug 19, 2026 Updated Aug 19, 2026 4 min read

Quick answer

Java Flight Recorder is a JDK event-recording framework that preserves detailed JVM and application evidence for later analysis or bounded streaming. Use Java 25 JFR to test hypotheses about CPU, allocation, garbage collection, locks, thread scheduling, I/O, and virtual-thread behavior. A recording is captured evidence, not a benchmark, a complete request population, or root-cause proof.

For the shared Spring order service, a JFR recording covers one selected JVM and time window; it must be correlated with user outcomes, dependencies, changes, and telemetry delivery evidence.

Begin with user impact and service signals. Capture only the duration, event set, stack depth, destination, and retention needed for the decision. Correlate timestamps, version, instance, route, and protected trace evidence without recording credentials or personal data.

Learning objectives

  • Use Java 25 Flight Recorder events to test CPU, allocation, GC, lock, I/O, and virtual-thread hypotheses with an explicit recording policy.
  • Correlate a bounded JFR recording with user, service, dependency, and change evidence without declaring one profile the root cause.

Prerequisites

Understand JVM threads, GC at a conceptual level, RED/USE signals, and Distributed Tracing and Context Propagation.

Evidence and system boundary

JFR records events inside one JVM plus selected operating-system observations. It does not see every database wait, broker state, network path, user retry, or durable order outcome. Java Mission Control and jfr commands analyze recordings; they do not change that evidence boundary.

Java 25 JFR is the stable required baseline. OpenTelemetry Profiles and other evolving profile protocols are mentioned only as Development boundaries and are not required course contracts.

java -XX:StartFlightRecording=name=order-incident,
duration=10m,settings=profile,filename=order-incident.jfr -jar order-service.jar

This is an explanatory command, not a production recommendation or executed result. A real team must approve settings, duration, storage, access, redaction, disk headroom, and rollback before capture.

Choosing evidence

CPU saturation calls for sampled execution, runnable-thread, compilation, and system-load evidence. Allocation or GC pressure calls for allocation samples, heap statistics, pause events, promotion behavior, and container memory. Lock hypotheses require monitor enter, park, thread state, and owner stacks. Virtual-thread incidents require carrier, pinning, scheduler, and downstream capacity evidence rather than thread count alone.

Event thresholds trade volume for detail. The default and profile configurations have different costs and event coverage. Custom application events can mark order-stage boundaries, but their fields require the same schema, privacy, and cardinality review as logs and traces.

Clock alignment matters. Preserve UTC timestamps, immutable service version, instance identity, and the capture window. A hot method after deployment is only correlated with the change until comparison against unaffected versions, requests, or instances supports the hypothesis.

Production failure scenario

Checkout latency rises while CPU stays moderate. A bounded JFR recording shows many order threads parked behind one synchronized cache refresh, with a small set of long monitor-enter events. Database pool wait also rises because blocked work retains connections.

The team does not call the lock the root cause from one recording. It compares affected traces, pool timing, deployment changes, lock owners, and a control instance. A reversible rollback removes the new critical section. Recovery requires fresh checkout SLI, pool wait, lock events, retries, and durable order-state evidence.

Common misconceptions

  • Low recording overhead is not zero overhead or permission to record indefinitely.
  • A flame graph shows captured samples, not total business impact.
  • High allocation does not automatically mean a memory leak.
  • Many virtual threads do not prove useful concurrency or downstream capacity.
  • One JVM recording cannot prove database, broker, network, or customer outcomes.

Decision checklist

  • State the user-impact hypothesis before recording.
  • Choose an approved bounded configuration and duration.
  • Verify disk, CPU, retention, and access limits.
  • Avoid secret, token, personal, and raw payload fields.
  • Preserve version, instance, environment, and time-window identity.
  • Compare JVM evidence with metrics, traces, logs, dependencies, and changes.
  • Use the smallest reversible mitigation.
  • Verify recovery outside the recording itself.

Previous: Distributed Tracing and Context Propagation. Continue the JVM evidence sequence with JVM Garbage Collection Logs and Pause Diagnosis, then use JVM Thread Dumps, Safepoints, and CPU Saturation Diagnosis and JVM JIT, Code Cache, and Warmup Diagnosis to test thread and compilation hypotheses. After that sequence, continue to OpenTelemetry Collector Pipeline. Follow the Production Observability & SRE course and topic cluster.

Use Production Java Concurrency Troubleshooting for thread and lock investigation, and Kubernetes Resource Requests, Limits, and JVM Containers for container memory and CPU evidence.

Official sources

Official sources accessed August 19, 2026. Recheck event availability and any Development profiling protocol before use.

Knowledge check

Check your understanding

Answer both questions correctly to mark this lesson as mastered. You can retry without penalty.

1. Checkout latency rises while CPU is moderate and a JFR recording shows long monitor contention; what conclusion is justified?

2. An operator needs production JFR evidence during a memory incident; what is the safe recording approach?

Keep reading