Lesson 14 — Testing Stage and ExEx — fixture chains, in-process nodes, golden state

Question

Testing patterns specific to Reth components. Fixture chains (small chains for stage tests), in-process node tests (full pipeline), golden state (deterministic reproducibility).

Principle (minimum model)

• Fixture chains. Hardcoded sequences of blocks for stage unit tests. ~10 blocks; fast; deterministic.
• TestStageDB. In-memory DB + injected blocks + assertions. Used for every stage test.
• In-process node. EthereumNode::test_instance().launch() spins up the whole pipeline in one process. Slower (~1s); used for integration tests.
• Golden state files. After running a stage on a fixture chain, save the resulting DB state to a file. Future tests assert against this file.
• Why golden state. Bug fixes that change semantics produce a different golden state. CI catches it; you review the diff.
• ExEx test harness. ExExTestHarness::new(ctx) lets you inject ExEx notifications and assert your ExEx behaviour.
• test_exex_context. Test-only ExExContext with pre-set backfill state. Lets you skip the real-Reth-boot overhead.
• Reorg simulation. Test harness can simulate reorgs by re-injecting blocks at an earlier height. Critical for ExEx correctness tests.

Worked example + steps

Testing Stage and ExEx — fixture chains, in-process nodes, golden state

You walked the Stage trait, the ExEx API, and the NodeBuilder SDK. Now: how does Reth — and the apps that extend it — verify any of this works? A node component that "looks correct" in dev silently corrupts state for thousands of users in production. The patterns below are how Reth's own CI guards against that, and how every ExEx-based app you ship in the Building tier needs to be tested.

The two test layers any Reth-extending app needs

Reth's own crates use both. Production extensions (tidx, MEV ExEx, custom App-chains) need both too. Skip either and a class of bugs gets through.

1. Stage unit testing — the canonical pattern

A Stage impl is two methods: execute (forward sync) and unwind (reorg rollback). Both are pure functions of the staged DB state at a given checkpoint. The unit test pattern: build a temp DB, seed pre-state, call execute, assert post-state; then call unwind and assert rollback.

use reth_provider::test_utils::create_test_provider_factory;
use reth_stages::test_utils::{TestRunnerError, TestStageDB};

#[tokio::test]
async fn execute_advances_checkpoint() {
    let db = TestStageDB::default();
    seed_blocks(&db, 0..=10).await;             // your fixture helper

    let mut stage = MyStage::default();
    let input = ExecInput { target: Some(10), checkpoint: None };
    let output = stage.execute(&db.factory.provider_rw().unwrap(), input).await.unwrap();

    assert_eq!(output.checkpoint.block_number, 10);
    assert!(output.done);
    assert_my_derived_state(&db, 10).await;
}

#[tokio::test]
async fn unwind_rolls_back_to_checkpoint() {
    let db = TestStageDB::default();
    seed_blocks(&db, 0..=10).await;
    let mut stage = MyStage::default();

    // Forward to 10
    stage.execute(&db.factory.provider_rw().unwrap(),
        ExecInput { target: Some(10), checkpoint: None }).await.unwrap();

    // Unwind to 5
    let output = stage.unwind(&db.factory.provider_rw().unwrap(),
        UnwindInput { unwind_to: 5, checkpoint: ..., bad_block: None }).await.unwrap();

    assert_eq!(output.checkpoint.block_number, 5);
    assert_my_derived_state(&db, 5).await;
}

The key tooling: create_test_provider_factory and TestStageDB (under reth_provider::test_utils and reth_stages::test_utils). They give you an ephemeral MDBX DB per test — no shared state, no cleanup boilerplate, no stale fixtures.

🔍 Find in repo. Open paradigmxyz/reth, search test_utils under crates/stages/. Read one of the existing stage tests (e.g., SenderRecoveryStage tests). Notice that every Reth-shipped stage has unit tests with this exact shape. Your custom stage should too.

2. ExEx unit testing — the harness pattern

reth-exex-test-utils exports a harness that lets you drive an ExEx with synthetic notifications without booting a full node:

use reth_exex_test_utils::{test_exex_context, PollOnce};
use reth_exex::{ExExEvent, ExExNotification};

#[tokio::test]
async fn handles_committed_then_reverted() {
    let (ctx, mut handle) = test_exex_context().await.unwrap();
    let exex = my_exex(ctx);
    tokio::spawn(exex);

    // Send a committed-chain notification covering blocks N..N+5
    handle.send_notification_chain_committed(committed_chain(N..=N+5)).await.unwrap();
    handle.assert_event_finished_height(N+5).await;

    // Reorg N+3..N+5
    handle.send_notification_chain_reverted(reverted_chain(N+3..=N+5)).await.unwrap();
    handle.assert_event_finished_height(N+2).await;

    // Verify your derived state
    assert_my_state_at_height(N+2).await;
}

The harness is what makes ExEx development tractable. Without it, you'd need to spin up a full node, replay real chain data, and wait for events — minutes per test cycle. With it, every notification is a single function call.

3. NodeBuilder integration testing — in-process Reth

Sometimes the unit-test layer isn't enough. If your code depends on the interaction between components (a custom pool builder reading from a custom payload validator, say), you need the assembled node:

use reth_node_builder::NodeBuilder;
use reth_node_ethereum::EthereumNode;
use reth_tasks::TokioTaskExecutor;

#[tokio::test]
async fn custom_pool_builder_filters_blob_txs() {
    let node = NodeBuilder::new(test_node_config())
        .testing_node(TokioTaskExecutor::default())
        .with_types::<EthereumNode>()
        .with_components(EthereumNode::components().pool(MyPoolBuilder))
        .with_add_ons(EthereumNode::add_ons())
        .launch()
        .await
        .unwrap();

    // Submit txs via the pool's API
    let pool = node.pool();
    let blob_tx = test_blob_tx();
    let result = pool.add_external_transaction(blob_tx).await;
    assert!(matches!(result, Err(PoolError::BlobsExcluded)));
}

The pattern: NodeBuilder::new(...).testing_node(...) boots the node in-process with your custom components, exposes handles (node.pool(), node.provider(), node.network()) that you can drive directly. Slower than unit tests (~1 second per test) but indispensable for cross-component behavior.

4. Fixture-chain testing — when canned data isn't enough

For the heaviest tests — full sync verification, multi-block reorg scenarios — you can replay real captured chain data:

#[tokio::test]
async fn full_sync_to_pinned_block_matches_golden_state() {
    let chain_fixture = load_fixture("tests/fixtures/sepolia_blocks_0_to_1000.rlp").await;
    let db = TestStageDB::default();

    // Drive every stage through the fixture
    for stage in default_stages_for_test() {
        run_stage_to_completion(&mut stage, &db, chain_fixture.range()).await;
    }

    // Compare derived state-root to the known-good value at block 1000
    let derived = db.factory.provider().header(1000).unwrap().state_root;
    assert_eq!(derived, GOLDEN_SEPOLIA_STATE_ROOT_AT_1000);
}

This is how Reth verifies sync correctness against historical data. The fixture is a one-time capture from a known-good node; CI replays it on every push. A regression in any stage shows up as a state-root mismatch — and that's a consensus bug, caught before mainnet.

How this connects to the Building tier

The Building tier Read a Real Production Indexer — tidx lesson lists this test gate:

Fixture chain replay — feed a known sequence of Notification::ChainCommitted / ChainReverted and assert your derived state matches a golden reference exactly.

That is §2 of this lesson at the application layer. The Reth-internal pattern (test harness + synthetic notifications) is exactly what the tidx test gate asks you to apply. When you reach Building, this lesson is the precondition.

Drill

1. Read one Reth stage test end-to-end. Open paradigmxyz/reth, find SenderRecoveryStage tests under crates/stages. Read one test top to bottom. Map every helper (TestStageDB, create_test_provider_factory, ExecInput) to where it appears in the test. 30 minutes.
2. Read one ExEx test using the harness. Search reth-exex-test-utils usages in the Reth repo and pick one. Trace what test_exex_context() returns and how the test drives notifications. 30 minutes.
3. Write a stage unit test from scratch. Take a trivial stage that just counts blocks (output.checkpoint.block_number = input.target). Test execute advances the checkpoint and unwind rolls it back. Use create_test_provider_factory. Get cargo test green. 1.5 hours.
4. Write an ExEx unit test from scratch. A trivial ExEx that increments a counter on each ChainCommitted and decrements on ChainReverted. Use test_exex_context() to drive 3 commits + 1 revert; assert counter ends at 2. 1 hour.
5. Run a NodeBuilder integration test. Pick the simplest test in crates/node/builder/ that uses testing_node(). Run it locally with cargo test. Read what the test asserts. 1 hour.

After drill 5, you can write tests for any custom Reth component you ship — no different from how the Reth maintainers test their own.

📺 Further reading

• reth_exex_test_utils — generated docs for the ExEx test harness
• Reth Book — Testing chapter — official guidance on stage and node tests

Expert continuation

Test discipline at the Reth-component level. At the systems level, two Expert lessons take this further:

• Differential fuzzing & execution-spec-tests — multi-implementation consensus-correctness testing
• Systems-code auditing — reading Reth patches with an auditor's eye

Summary (3 lines)

• Fixture chains + TestStageDB for stage tests. In-process EthereumNode::test_instance for integration. Golden state files for reproducibility.
• ExExTestHarness + test_exex_context for ExEx tests. Reorg simulation for correctness.
• Production CI runs all of these. Pattern transfers to any Reth-based codebase.