ProductMarch 31, 202610 min read

Continuous Profiling and Flame Graphs: Find Performance Bottlenecks

Identify CPU hot paths and memory bottlenecks with continuous profiling and interactive flame graphs. No code changes, no performance impact, always on.

The Missing Piece in Performance Monitoring

APM tells you which requests are slow. Distributed tracing tells you where in the call chain the slowness happens. But neither tells you why the code is slow at the CPU instruction level. That's what profiling is for.

Traditionally, profiling has been a developer's local tool. You attach a profiler to a running process, reproduce the problem, analyze the output, and hope the issue occurs in your local environment. Continuous profiling changes this model entirely. It samples your production processes continuously, aggregates the data, and presents it as interactive flame graphs that reveal exactly where your CPU time and memory are being spent.

JustAnalytics now includes continuous profiling for Node.js services, integrated directly with your traces, errors, and logs.

How Continuous Profiling Works

Continuous profiling uses statistical sampling to capture stack traces at regular intervals (typically every 10ms) with minimal overhead. Over time, these samples build a statistically accurate picture of where your application spends its time.

CPU Profiling

CPU profiling answers the question: where is my application spending CPU cycles?

Every 10 milliseconds, the profiler captures the current call stack of every active thread. After thousands of samples, patterns emerge:

Total CPU time: 60 seconds

Function                          | Self Time | Total Time | Samples
----------------------------------|-----------|------------|--------
json.parse                        | 12.4s     | 12.4s      | 1,240
prisma.query                      | 8.2s      | 15.6s      | 820
middleware.authCheck               | 3.1s      | 4.8s       | 310
zlib.deflateSync                  | 6.7s      | 6.7s       | 670
express.router.handle             | 1.2s      | 58.3s      | 120

In this example, json.parse consumes 20% of total CPU time -- a clear optimization target. Perhaps you're parsing large JSON payloads that could be streamed, or parsing the same data multiple times.

Memory Profiling

Memory profiling shows where allocations happen and helps identify memory leaks:

Allocation Site                   | Allocations | Size       | Retained
----------------------------------|-------------|------------|----------
Buffer.from (response handler)    | 45,000/min  | 892 MB     | 12 MB
new Object (ORM hydration)       | 120,000/min | 340 MB     | 340 MB
Array.push (log buffer)          | 8,000/min   | 156 MB     | 156 MB
String.concat (template render)  | 200,000/min | 89 MB      | 2 MB

The "Retained" column is key. If retained memory keeps growing, you have a leak. In this example, the ORM is retaining 340 MB of hydrated objects -- likely because query results aren't being garbage collected properly.

Enabling Profiling

Using the JustAnalytics Node.js SDK

Profiling is built into the Node.js SDK. Enable it with a single configuration flag:

import { init } from '@justanalyticsapp/node';

init({
  apiKey: 'ja_live_abc123',
  serviceName: 'my-api',
  profiling: {
    enabled: true,
    sampleRate: 100,        // Sample every 100ms (default: 10ms)
    cpuProfiling: true,
    memoryProfiling: true,
    wallTimeProfiling: true, // Includes I/O wait time
  },
});

Using the OpenTelemetry SDK

If you're using OpenTelemetry directly, add the JustAnalytics profiling exporter:

import { ProfilingExporter } from '@justanalyticsapp/node/profiling';

const exporter = new ProfilingExporter({
  endpoint: 'https://ingest.justanalytics.app/v1/profiles',
  apiKey: process.env.JUSTANALYTICS_API_KEY,
});

Performance Overhead

Continuous profiling is designed to run in production with negligible overhead:

SettingCPU OverheadMemory Overhead
CPU profiling (10ms)< 1%~5 MB
CPU profiling (100ms)< 0.1%~2 MB
Memory profiling< 0.5%~8 MB
All profiling enabled< 2%~15 MB

The overhead comes from the sampling itself and the periodic export of profile data. At the default 10ms sampling interval, the profiler adds less than 1% CPU overhead -- well within acceptable limits for production.

Reading Flame Graphs

Flame graphs are the primary visualization for profiling data. They were invented by Brendan Gregg at Netflix and have become the standard way to understand where CPU time is spent.

Anatomy of a Flame Graph

A flame graph is read from bottom to top:

┌─────────────────────────────────────────────────────────┐
│                     json.parse (20.7%)                   │
├──────────────────────────────┬──────────────────────────┤
│    parseObject (12.3%)       │    parseArray (8.4%)     │
├──────────────────────────────┴──────────────────────────┤
│              handleRequest (95.2%)                        │
├─────────────────────────────────────────────────────────┤
│               express.router (98.7%)                     │
├─────────────────────────────────────────────────────────┤
│                    main (100%)                            │
└─────────────────────────────────────────────────────────┘
  • Width represents the proportion of total time spent in that function
  • Height represents call stack depth
  • Color in JustAnalytics represents the package (your code vs dependencies vs runtime)
  • Hover to see exact sample counts and percentages
  • Click to zoom into a subtree

Interactive Features

JustAnalytics flame graphs are fully interactive:

  • Search -- highlight all frames matching a function name or pattern
  • Zoom -- click any frame to zoom in and see its subtree at full width
  • Filter -- show only frames from your code, a specific package, or the runtime
  • Time range -- narrow the profile to a specific time window
  • Compare -- overlay two profiles to see what changed (differential flame graph)

Aggregated Flame Graphs

Individual profiles are useful, but the real power comes from aggregation. JustAnalytics aggregates profile samples across all instances of a service over time, giving you a statistically significant view of where CPU time is spent in production.

Why Aggregation Matters

A single 60-second profile might show an anomalous code path that was triggered by an unusual request. An aggregated profile over 24 hours shows you the true steady-state behavior of your service.

Aggregated profiles answer questions like:

  • What are the top CPU consumers across my entire fleet?
  • Has the CPU profile of my service changed after the last deployment?
  • Which functions account for the most CPU time over the past week?

Filtering Aggregated Profiles

You can filter aggregated profiles by:

  • Service -- compare profiles across services
  • Environment -- production vs staging
  • Version -- compare CPU profiles across releases
  • Endpoint -- profile only requests to a specific route
  • Time range -- any window from 15 minutes to 30 days

Differential Flame Graphs

Differential flame graphs are the killer feature for performance debugging. They compare two profiles and highlight what changed.

How They Work

Select two time ranges (or two versions) and JustAnalytics generates a differential flame graph:

  • Red frames -- these functions use more CPU time in the second profile
  • Blue frames -- these functions use less CPU time in the second profile
  • Gray frames -- no significant change

Use Cases

After a deployment: Compare the profile from before and after your latest release. If a function suddenly appears in red, your new code introduced a performance regression.

Comparing: v2.3.0 → v2.4.0

Functions with increased CPU time:
  +340%  newSerializer.transform (added in v2.4.0)
  +120%  json.stringify (called more frequently)
  +45%   database.query (new N+1 query pattern)

Functions with decreased CPU time:
  -60%   oldSerializer.transform (replaced)
  -25%   cache.get (better cache hit rate)

Before and after an optimization: Verify that your performance fix actually worked. If the target function turns blue, your optimization was effective.

During an incident: Compare the current profile with a baseline from a healthy period. The differential view immediately shows what's different about the current execution pattern.

Connecting Profiles to Traces

In JustAnalytics, profiling data is linked to your distributed traces. When you view a trace in the Trace Explorer and notice a slow span, you can click View Profile to see the flame graph for exactly that time period and service.

Span-Level Profiling

For any span in a trace, JustAnalytics can show:

  • CPU flame graph scoped to the span's duration
  • Top functions that consumed the most CPU during the span
  • Memory allocations that occurred during the span

This closes the debugging loop completely:

  1. APM dashboard shows you that p99 latency increased
  2. Trace Explorer shows you which span is slow
  3. Flame graph shows you which function in that span is the bottleneck
  4. You fix the function and deploy
  5. Differential flame graph confirms the fix worked

Identifying Common Hot Paths

Over time, JustAnalytics builds a catalog of hot paths in your application. A hot path is a code path that consistently consumes a significant proportion of CPU time.

Hot Path Reports

The hot path report ranks functions by their total CPU time across all profiles:

Top CPU Consumers (Last 7 Days)

Rank | Function                    | CPU %  | Trend
-----|-----------------------------|--------|-------
  1  | json.parse                  | 18.4%  | ↑ 2.1%
  2  | prisma.queryRaw             | 12.7%  | → stable
  3  | zlib.deflateSync            | 9.3%   | ↓ 1.5%
  4  | bcrypt.hashSync             | 7.8%   | → stable
  5  | ejs.compile                 | 6.2%   | ↑ 0.8%

Optimization Suggestions

For common hot paths, JustAnalytics provides context-aware suggestions:

  • json.parse at 18% -- "Consider streaming JSON parsing for large payloads, or caching parsed results"
  • bcrypt.hashSync at 7.8% -- "Switch to bcrypt.hash (async) to avoid blocking the event loop"
  • zlib.deflateSync at 9.3% -- "Switch to zlib.deflate (async) or use a streaming compression approach"

These aren't generic tips. They're based on the actual call sites and patterns in your profiling data.

Setting Up Profiling

Step 1: Enable in SDK

import { init } from '@justanalyticsapp/node';

init({
  apiKey: 'ja_live_abc123',
  serviceName: 'my-api',
  profiling: { enabled: true },
});

Step 2: Deploy

Restart your service with profiling enabled. Profile data starts flowing immediately.

Step 3: View Flame Graphs

Navigate to Monitoring > Profiling in your JustAnalytics dashboard. Select your service and time range. The flame graph renders within seconds.

Step 4: Compare Releases

After your next deployment, use the differential flame graph to compare performance before and after. This should become a standard part of your deployment checklist.

Performance Profiling Best Practices

Start with the Biggest Bars

In a flame graph, the widest bars represent the most CPU time. Start there. Optimizing a function that accounts for 18% of CPU time will have far more impact than optimizing one at 0.3%.

Profile in Production

Local profiling is useful for micro-benchmarks, but production profiling captures real traffic patterns, real data sizes, and real concurrency levels. What's fast in development might be slow under production load.

Use Differential Flame Graphs for Every Release

Make it a habit. Before merging a PR, compare the staging profile with the production baseline. Catch performance regressions before they reach users.

Set CPU Budget Alerts

JustAnalytics can alert you when a function's CPU share exceeds a threshold:

Alert: CPU Budget Exceeded
Condition: Function "json.parse" exceeds 25% of total CPU time
Window: 1 hour
Notify: Backend team

This catches gradual performance degradation that might otherwise go unnoticed.

Pricing

Continuous profiling is included in all JustAnalytics plans:

PlanProfile retentionSamples/month
Free3 days10M
Pro14 days100M
Business30 days1B
EnterpriseCustomCustom

No per-host pricing. No hidden costs. Just enable it and go.

Start your 7-day free trial and find your next performance bottleneck.

JT
JustAnalytics TeamEngineering Team

The engineering and product team behind JustAnalytics. We're on a mission to make web observability simpler, faster, and more private.

Related posts