Lesson 6 — Building the ExEx API step by step

Question

ExEx = "Execution Extension" = Reth's plug-in for indexing / observing chain state from inside the node. Build the API from scratch.

Principle (minimum model)

• Step 0–1 — polling, then a callback. A separate process polling RPC lags, races Reth's commits, and sees reorgs second-hand. A per-block callback fixes latency but can't express reorgs.
• Step 2 — three notification variants. enum ExExNotification { ChainCommitted { new }, ChainReorged { old, new }, ChainReverted { old } }. Production needs all three arms.
• Step 3 — ExExEvent::FinishedHeight. Tells Reth what you've processed; the pruner and the ExEx WAL never advance past the lowest one.
• Step 4 — stream pull. ctx.notifications.try_next().await, not a callback: you consume at your own pace, in order.
• Step 5 — init/run split. exex_init does setup and returns the long-running future. An init error fails node startup; a future that returns at all stops the node.
• Step 6 — install_exex(name, init). Register several ExExs on one node, each with its own stream and its own FinishedHeight.

Worked example + steps

Building the ExEx API step by step

ExEx (Execution Extension) is Reth's mechanism for injecting Rust code into the execution loop. With it you build node-speed indexers, MEV bots, and live risk engines — directly in the same process as the chain itself.

But the API has 4 parts that look weighty: an init/run split, a notification enum with 3 variants, an event channel for pruning hints, and an install method on the node builder. Walk it cold and you get four ideas at once.

This lesson builds the API up from the simplest possible "block listener." By the end you'll have built every piece of the real minimal ExEx — which the next lesson reads in detail.

📂 Open paradigmxyz/reth-exex-examples/minimal in another tab. That's the file we're building toward.

Step 0 — The naive indexer: separate process polling RPC

Without thinking, you'd index Ethereum like this:

fn main() {
    let rpc = HttpProvider::new("http://localhost:8545");
    let mut last_block = 0;
    loop {
        let head = rpc.get_block_number().unwrap();
        for n in (last_block+1)..=head {
            let block = rpc.get_block(n).unwrap();
            index(block);
        }
        last_block = head;
        sleep(Duration::from_secs(1));
    }
}

A separate process. Polls the RPC every second. Indexes any new blocks.

The three:

1. Latency. The RPC poll has request/response overhead. The indexer is always seconds behind the tip — useless for MEV, risk, real-time UX.
2. Atomicity. Reth commits a new block to disk before your indexer sees it. There's a window where Reth has a block your code hasn't processed. If your code is the source of truth for a derived view, that window is a race condition.
3. Reorgs. Polling sees head = N, then later head = N (different block). Your indexer has to detect and handle reorgs from outside, with weaker information than Reth itself has.

The fix: run in the same process as Reth. Get notified the moment a block commits, with full chain context.

Step 1 — First stab: a callback per block

Naive in-process API:

fn on_new_block<F: Fn(&Block)>(reth: &mut Reth, callback: F) {
    reth.add_listener(callback);
}

Reth calls your closure for every new block. Simple.

The two:

1. Reorgs aren't append-only. A callback that only fires on "new block added" can't represent "block N at hash X was replaced by block N at hash Y." Your indexer's derived state silently corrupts on every reorg.
2. No way to tell Reth you're done. If your indexer is processing block N, Reth doesn't know whether it can prune block N-100,000's data. Without this signal, Reth keeps everything forever.

Step 2 — A richer notification: the three chain events

Replace the bare callback with an enum that captures all three things that can happen to the chain:

enum ExExNotification {
    ChainCommitted { new: Chain },             // canonical blocks added
    ChainReorged   { old: Chain, new: Chain }, // old replaced by new
    ChainReverted  { old: Chain },             // removed (no replacement yet)
}

Each variant carries enough information for the indexer to undo or redo derived state:

• ChainCommitted { new } — append the new blocks' state to your index.
• ChainReorged { old, new } — undo old's state, apply new's state. Atomic swap.
• ChainReverted { old } — undo old's state, wait. Reth will follow up with a future ChainCommitted once it picks a new tip.

The HashMap contains the old chain's transactions, but the canonical chain is now the new chain. Any later read off your index will return transactions that no longer exist on the canonical chain — a phantom-data bug. Worse: the new chain's transactions never got indexed (you didn't see a ChainCommitted for them — you saw a ChainReorged you ignored).

This is the #1 ExEx bug. The three-variant enum exists specifically to prevent it.

Step 3 — Tell Reth what you've finished: FinishedHeight

If your indexer has processed block N, Reth needs to know. Otherwise it can't safely prune anything below N.

ctx.events.send(ExExEvent::FinishedHeight(block_number_hash))?;

ctx.events is a write-only channel back to Reth. Whenever your handler finishes a block (or chain), you send FinishedHeight(N). Reth aggregates the minimum across all installed ExExes and prunes below that.

The node retains all historical state Reth would otherwise prune — because it can't safely prune anything your ExEx might want to read later. Disk usage compounds. Without FinishedHeight, an "innocuous indexer" turns Reth into a full-archive node by accident.

Step 4 — The notification stream: async pull, not callback

A callback would force Reth to wait for your slow code on every block. Better: push notifications into a stream, your handler pulls when ready:

while let Some(notification) = ctx.notifications.try_next().await? {
    // process at your pace
}

ctx.notifications is a Stream of ExExNotification. try_next is async — your handler runs in the same async runtime as Reth. Reth doesn't wait for your handler on each block: notifications are buffered (up to the ExEx manager's capacity), and your handler still observes every event in order.

When Reth shuts down or the channel closes, the loop exits cleanly with Ok(()).

Step 5 — The init/run split

A user wants to do synchronous setup (open files, init a database, allocate a buffer) before the async loop starts. A single async fn would force this setup into the future itself, where it can't be reasoned about clearly:

async fn exex_init<Node: FullNodeComponents>(
    ctx: ExExContext<Node>,
) -> eyre::Result<impl Future<Output = eyre::Result<()>>> {
    // synchronous setup goes here
    Ok(exex(ctx))  // return the long-running future
}

Two functions:

• exex_init — runs once at node startup. Synchronous setup. Returns a future.
• exex (the future) — runs forever (or until shutdown). Polls the notification stream.

Reth needs to acknowledge the ExEx is alive before it starts pushing notifications. If you put File::open(...) inside exex, the file open happens after Reth has already started buffering notifications — and if it fails (permissions, missing path), notifications pile up while Reth thinks the ExEx is healthy. The init/run split lets Reth distinguish "ExEx couldn't start" from "ExEx ran for a while and crashed."

Step 6 — install_exex: multiple extensions

Your main wires the ExEx into the node builder:

.install_exex("MyIndexer", exex_init)
.install_exex("MevWatcher", mev_init)
.install_exex("RiskEngine", risk_init)

The first arg is a name (used in metrics and logs); the second is the init function. You can chain multiple .install_exex(...) calls — each ExEx gets its own notification stream and its own FinishedHeight channel.

Practical implications:

• Mostly independent, with a shared buffer. Each ExEx has its own notification stream, so a lagging indexer doesn't delay the MEV watcher's next event. They do share the ExExManager's buffer, though: if one ExEx is slow for long enough to fill it, nobody can send new notifications until there is room again (crates/exex/exex/src/manager.rs:268-272).
• The pruner aligns to the slowest one. The pruner only prunes below the minimum FinishedHeight across all installed ExExs. If the indexer stalls at block 100, Reth keeps everything after 100, even if the MEV watcher is at 1000.
• Crashes are not isolated. Each ExEx runs as a critical task. If its future returns, with Ok or Err, Reth panics with "ExEx … finished" / "ExEx … crashed" and the node shuts down (crates/node/builder/src/launch/exex.rs:124-131). An error in exex_init fails node startup.
• Metrics are per-ExEx. The first-arg name becomes the exex label on the reth_exex_* metrics (notifications_sent_total, events_sent_total).

So a dead ExEx is loud: the node stops with it. The quiet disk killer is the ExEx that is alive but stuck, or one that never sends FinishedHeight. The pruner stops at its last FinishedHeight, and so does ExEx-WAL cleanup. Every later block is kept because the ExEx might still need it. Once the WAL grows past its warning threshold, the manager logs "WAL contains too many blocks and is not getting cleared… Check that you emit the FinishedHeight event from your ExExes" (crates/exex/exex/src/manager.rs:395-415). History keeps piling up until the ExEx catches up, so alert on that log line and on a FinishedHeight that stops moving.

What you've built

Every piece earned its keep:

• ExExNotification enum with 3 variants (Step 2) — handles append, reorg, revert
• FinishedHeight event (Step 3) — opt-in pruning, prevents disk bloat
• Stream-pulled notifications (Step 4) — Reth doesn't block on your handler
• init/run split (Step 5) — synchronous setup before async loop
• install_exex (Step 6) — multiple extensions, each with its own stream

The next lesson reads the minimal ExEx — ~40 lines of main.rs — and shows how all five pieces fit together in real code.

Recall before moving on

Without scrolling:

1. Why does the API push notifications via a stream instead of calling your code directly?
2. What are the three ExExNotification variants, and why do you need all three?
3. What does FinishedHeight tell Reth, and what's the disk consequence of forgetting it?
4. Why is the API split into exex_init (sync) and exex (async future)?

If any answer is shaky, scroll back. The next lesson reads the real minimal ExEx in detail.

🧭 Where you are now in the stack: you've built the database layer's pub-sub + prune protocol — 3-variant ExExNotification, FinishedHeight backpressure, stream pull, init/run split, install_exex injection. Same shape as Kafka consumer offset management and Postgres logical replication slots, applied to EVM chain sync. Next lesson reads the minimal ExEx's real source against this model.

Summary (3 lines)

• Buildup: RPC polling → callback → 3-variant ExExNotification → FinishedHeight → stream pull → init/run split → install_exex.
• Production-grade ExEx indexers use the full pattern. A stuck ExEx holds back pruning; a returning one stops the node.
• Next: walkthrough the minimal ExEx.