The JIT: speculative optimization and deoptimization

Lesson 1 named the shape of V8's pipeline: interpret everything cheaply in Ignition, then compile the hot parts to machine code with TurboFan. This lesson opens up that second step, because "compile the hot parts" hides the actual mechanism, and the mechanism is a bet. TurboFan doesn't compile your function the way a C compiler compiles a function — it compiles the function as it has behaved so far, wagers that behavior will continue, and builds in a way to back out cleanly the moment the wager loses. That combination — speculate, then be ready to deoptimize — is the JIT.

Profiling: the evidence the bet is based on

Ignition doesn't just interpret bytecode, it watches itself do it. Every time an operation like a + b or a property access like obj.x executes, Ignition records what it saw — the types of the operands, the shape of the object — into a feedback structure attached to that specific spot in the bytecode. Alongside that, it counts invocations: how many times has this function been called, how many times has this loop iterated. This is type feedback, and it accumulates for free as a side effect of running the interpreter — no separate profiling pass, no instrumentation build.

Once a function's call count (or a loop's iteration count) crosses V8's internal "hotness" threshold, the function is queued for optimization. Ordinary functions get promoted after enough calls; a function containing a very long-running loop can be promoted before it even returns, through a mechanism called on-stack replacement (OSR) — V8 compiles an optimized version and swaps the currently-executing loop over to it mid-iteration, rather than waiting for the function to be called again from scratch. OSR exists because "this function is hot" and "this function has returned so we can safely re-enter it optimized" are different events, and a long loop can satisfy the first for a long time before it satisfies the second.

TurboFan: compiling the evidence, not the source

When TurboFan picks up a hot function, it doesn't start from the AST — it uses the accumulated type feedback to decide what machine code to emit. If every recorded sample of a + b at that bytecode offset saw two small integers, TurboFan doesn't emit the general "figure out what + means for whatever these turn out to be" logic (check if either is an object needing valueOf, check for string concatenation, check for overflow into a float, ...). It emits a specialized integer add — a handful of machine instructions — because the feedback says that's the only case that has ever actually occurred here.

This is where the speed comes from. The general case of almost any JavaScript operation is slow, because the language permits so much. The specific case observed at a specific call site is usually narrow and fast. TurboFan's whole value proposition is: skip the general case and go straight to the specific one, everywhere the profile supports it.

The bet is speculative, so it has to be guarded

But "every sample so far was an integer" is a statement about the past, not a guarantee about the future. Nothing in JavaScript stops the next call from passing a string. So TurboFan can't simply emit the specialized integer add and walk away — it has to emit the specialized code plus a cheap check in front of it: is this value still a number? Does this object still have the hidden class TurboFan compiled against? These are guards, and they're deliberately cheap — a type tag comparison, a hidden-class pointer comparison — because they run on every single execution of the optimized code, including all the ones where the bet keeps paying off.

As long as the guard holds, execution stays in the fast, specialized path and none of the general-case machinery ever runs. The guard is the price of speculating; it's small precisely so that speculating stays worth it.

Deoptimization: cashing out the bet when it loses

When a guard fails — the function that has only ever seen numbers is handed a string — the optimized machine code is no longer valid for this call, and V8 has to deoptimize. This is more involved than just jumping to a different function pointer, because the optimized code was running with values in registers, computations reordered, and whole checks eliminated that the interpreter's bytecode still expects to see performed step by step. V8 has to reconstruct the exact state Ignition would have been in at this point in the function — which bytecode offset, which local variables, which values on the interpreter's own stack — from the optimized code's very different internal state, then resume execution there in the interpreter. This reconstruction is the expensive part of deopt: it's not the fallback path being slow to run, it's the cost of rebuilding the world the slow path expects.

That expense is why deopt is something to avoid triggering repeatedly rather than a free safety net. A function that deoptimizes once, gets reoptimized, and deoptimizes again on the same kind of instability falls into deopt/reopt thrashing — V8 keeps paying to compile it, throwing the result away almost immediately, and paying again. Past a certain number of deopts at the same location, V8 gives up and marks that function as unfit for optimization for a while, which means it's stuck running in the interpreter — often slower, in aggregate, than if it had never been "hot" enough to attract TurboFan's attention in the first place.

Monomorphic, polymorphic, megamorphic

V8 has names for how many distinct shapes or types a single call site or property access has actually seen, and they matter because they predict how well TurboFan can specialize:

• Monomorphic — the site has only ever seen one shape or type. This is the best case: TurboFan emits one specialized path, guarded by one cheap check.
• Polymorphic — the site has seen a handful of distinct shapes or types (V8 tracks a small fixed number, typically up to four). TurboFan can still specialize, but now it emits a short chain of guarded cases to check against, which is slower than the monomorphic single check but still far better than a fully generic path.
• Megamorphic — the site has seen more shapes or types than V8 is willing to track individually. At this point V8 gives up on a per-shape fast path altogether and falls back to a generic, slower lookup for that site, because the cost of maintaining and checking a growing list of guards would exceed the cost of just doing the general-purpose thing.

A hot loop that always adds two numbers is monomorphic and TurboFan loves it. A hot loop that sometimes adds numbers and sometimes concatenates strings — even if both cases are individually fast — pushes that call site toward polymorphic or megamorphic, and every optimization pass has to hedge against the shapes actually seen instead of committing to one.

What actually triggers a deopt

In practice, the deopt triggers you'll run into are all instances of one theme — reality stopped matching the profile:

• A variable's type changes. A function parameter or local that was always a number on every prior call now arrives as a string, null, or undefined.
• An object's shape changes. An object TurboFan compiled against as "has properties x and y at these offsets" gets a new property added, or a property deleted, after the optimized code was already compiled against the old shape. (Lesson 5 covers why shape is a first-class concept at all.)
• Inconsistent argument types across calls. A function called with numbers a thousand times, then called once with an object, forces a guard failure on that one odd call — and if that pattern repeats, it forces repeated deopts.

A function that gets optimized, then deopts

function add(a, b) {
  return a + b;
}

// Thousands of calls, always with numbers — Ignition's feedback
// says "a and b are always numbers." V8 eventually compiles `add`
// with TurboFan, specialized for a fast numeric add, guarded by a
// cheap "are these still numbers?" check.
for (let i = 0; i < 100000; i++) {
  add(i, i + 1);
}

// Now the guard sees something it didn't expect.
add("5", "6"); // guard fails -> deoptimize -> back to Ignition for this call

You can watch this happen for real by running Node with --trace-deopt (and --trace-opt to see the optimization side too). V8 will print a line naming the function, the bytecode offset, and the deopt reason (something like "wrong call target" or a type mismatch) the moment the guard trips — which turns "the JIT deoptimized" from a theoretical claim into a line of output you can point at.

The loop this whole lesson describes

Profile in Ignition, optimize in TurboFan based on that profile, guard the optimization's assumptions, deoptimize back to Ignition the moment a guard fails, and — because Ignition is still profiling during that fallback execution — start feeding fresh feedback for a possible future reoptimization. It's a loop, not a one-shot pipeline stage, and it runs continuously for every function in a long-lived program: hot code gets speculated on again and again, and each deopt is itself new information about what to speculate on next time.

This is the sharpest possible contrast with WebAssembly: a WASM module's types are declared statically and validated before the module runs at all, so there's no profiling stage, nothing to speculate about, no guards, and no deopt path to fall back through — the compiler that produces WASM's machine code is right by construction, not right until proven wrong. JavaScript's JIT is fast despite not having that guarantee, by building an entire apparatus — feedback, speculation, guards, deopt — to approximate it at runtime instead of compile time.

Where this goes next

Everything TurboFan speculates on ultimately traces back to one question asked over and over: does this object still look the way it looked last time? Hidden classes and inline caches is about how V8 answers that question cheaply — giving every object a hidden layout descriptor and caching lookups against it — which is the mechanism the "object shape changed" deopt trigger above is quietly depending on.