Introduction

Go 1.25 introduces a powerful new diagnostic tool: the flight recorder. It builds on the execution tracing capabilities introduced earlier, but solves a key problem for long-running services: how to capture trace data after a problem occurs. Traditional execution traces require you to start recording before the issue happens, which is impractical for web servers that run for days. The flight recorder continuously buffers the last few seconds of execution trace data in memory, so when your application detects an error or timeout, you can snapshot that exact window and analyze the root cause. This guide walks you through setting up and using the flight recorder in your Go application.

What You Need

• Go 1.25 or later – The flight recorder is available starting with Go 1.25. Verify your version using go version.
• An existing Go application – Preferably a long-running service (HTTP server, gRPC server) where you want to debug intermittent latency or failures.
• Basic familiarity with Go execution traces – Understanding of runtime/trace package is helpful but not required.
• A development environment with the go toolchain installed.

Step-by-Step Guide

Step 1: Import the runtime/trace Package

Start by importing the standard runtime/trace package into your application. This package provides the flight recorder functionality alongside traditional tracing.

import "runtime/trace"

Step 2: Enable the Flight Recorder

Instead of calling trace.Start() which writes to a file, you enable flight recording using trace.StartFlightRecorder(). This starts buffering execution trace events in memory, keeping only the most recent few seconds (configurable via the duration parameter).

// Start flight recorder with a 10-second buffer
if err := trace.StartFlightRecorder(10 * time.Second); err != nil {
    log.Fatalf("failed to start flight recorder: %v", err)
}
defer trace.StopFlightRecorder()

The buffer size determines how far back the recorded data goes. Choose a duration that covers the typical window between when a problem occurs and when your application detects it.

Step 3: Trigger a Snapshot When a Problem Occurs

When your application detects an error, timeout, or any abnormal condition, call trace.SnapshotFlightRecorder() to retrieve the buffered trace data. This returns an io.Reader containing the execution trace of the last few seconds leading up to the snapshot moment.

// Inside your error handler or health check failure
if err != nil {
    snapshot := trace.SnapshotFlightRecorder()
    // Write snapshot to a file or send to a logging system
    f, _ := os.Create("/tmp/crash.trace")
    io.Copy(f, snapshot)
    f.Close()
    log.Println("Flight recorder snapshot saved to /tmp/crash.trace")
}

You can take multiple snapshots over time – each call returns the current buffer without resetting it. The flight recorder continues buffering after a snapshot.

Step 4: Analyze the Trace Data

Use the go tool trace command to analyze the snapshot file. This visualizes goroutine activity, network blocking, garbage collection, and more.

go tool trace /tmp/crash.trace

This opens a web interface in your browser. Look for:

• Goroutines that are blocked for long periods (red sections in the timeline).
• Network or system calls that take unusually long.
• Synchronization issues like mutex contention.

Step 5: Integrate the Flight Recorder into Your Service

For production use, embed the flight recorder in your server startup. For example, in an HTTP server’s main() function:

func main() {
    // Start flight recorder with 15-second buffer
    trace.StartFlightRecorder(15 * time.Second)
    defer trace.StopFlightRecorder()

    http.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) {
        // Simulate a health check that might fail
        if someCondition {
            snapshot := trace.SnapshotFlightRecorder()
            // Write snapshot to a file named with timestamp
            filename := fmt.Sprintf("/tmp/trace-%d.trace", time.Now().Unix())
            f, _ := os.Create(filename)
            io.Copy(f, snapshot)
            f.Close()
            http.Error(w, "Service unhealthy", 503)
            return
        }
        w.WriteHeader(http.StatusOK)
    })

    log.Fatal(http.ListenAndServe(":8080", nil))
}

You can also save snapshots to a remote storage (e.g., S3) for centralized analysis.

Tips for Effective Flight Recording

• Choose the right buffer size – A longer buffer uses more memory but captures more context. For most services, 5-15 seconds is sufficient.
• Don’t overuse snapshots – Each snapshot copies the buffer, which could impact performance if done too frequently. Only snapshot when you need to debug a specific incident.
• Combine with logging – Include a unique identifier (e.g., request ID) in both your logs and the snapshot filename to correlate traces with error logs.
• Test in staging first – Validate that the flight recorder doesn’t cause any unexpected memory pressure or performance degradation in your production-like environment.
• Use for both failures and near-failures – The flight recorder is useful even if your service doesn’t crash. For example, capture a snapshot when a request takes longer than a threshold to identify hidden bottlenecks.
• Automate trace analysis – Consider writing scripts to analyze snapshot files programmatically using go tool trace -trace=... or by parsing the binary format (though undocumented).

The Go 1.25 flight recorder gives you a surgical tool for capturing exactly the trace data you need, when you need it. By following these steps, you can diagnose latency issues and failures that previously were nearly impossible to reproduce in production.