Lesson 1 — Building the Stage trait step by step

Question

Build the Stage trait from scratch. Naive single-pass sync is wrong (no resumability, no checkpoints, no parallelism). Six steps add what production needs.

Principle (minimum model)

• Step 0 — naive. async fn sync_block_to_state(block). No checkpoint; if it crashes halfway, restart from scratch.
• Step 1 — checkpoint. Persist progress after each batch; resume from there.
• Step 2 — backward direction. Some stages run forward; some need backward (rollback on reorg). Two methods.
• Step 3 — Result with done flag. Stage may complete part of work; return (progress, done).
• Step 4 — generic over DB. DB type as a generic; tests can use in-memory; production uses MDBX.
• Step 5 — Send + Sync + 'static. Required for Tokio task spawning.
• Step 6 — 6 trait methods. id + execute + unwind + execute_progress + unwind_progress + post_unwind. Each has a specific role.

Worked example + steps

Building the Stage trait step by step

Staged Sync is the spine of Reth. It also looks intimidating — the real Stage trait has 6 methods, async readiness, two-direction symmetry, and an auto_impl(Box) attribute. Walk it cold and you get six new ideas at once.

This lesson builds the trait up from the simplest possible sync loop. By the end you'll have built every piece of:

#[auto_impl::auto_impl(Box)]
pub trait Stage<Provider>: Send {
    fn id(&self) -> StageId;
    fn poll_execute_ready(&mut self, _cx: &mut Context<'_>, _input: ExecInput)
        -> Poll<Result<(), StageError>> { Poll::Ready(Ok(())) }
    fn execute(&mut self, provider: &Provider, input: ExecInput)
        -> Result<ExecOutput, StageError>;
    fn post_execute_commit(&mut self) -> Result<(), StageError> { Ok(()) }
    fn unwind(&mut self, provider: &Provider, input: UnwindInput)
        -> Result<UnwindOutput, StageError>;
    fn post_unwind_commit(&mut self) -> Result<(), StageError> { Ok(()) }
}

📂 Open paradigmxyz/reth in another tab. Cross-check at every step.

Step 0 — The naive sync: block by block

Without thinking, you'd write Ethereum sync as:

fn sync_to_tip(client: &mut RethNode) -> Result<(), Error> {
    while let Some(block) = client.next_block()? {
        let header = client.fetch_header(block)?;
        let body   = client.fetch_body(block)?;
        let senders = recover_senders(&body)?;
        let receipts = client.execute(&block, &header, &body)?;
        client.update_state(receipts)?;
        client.update_merkle_root(&block)?;
        client.write_indexes(&block)?;
        client.commit()?;
    }
    Ok(())
}

One block at a time. Each block goes through every phase before the next block starts.

The three:

1. No batching. ECDSA sender recovery is the same operation 200 times per block. Doing it in 200 separate calls is 200 separate setup costs.
2. No I/O amortization. Writing one Merkle root per block means 20M commit() calls — each touches disk. Batched, you write Merkle roots once per million blocks.
3. No parallelism. Headers don't depend on tx execution; sender recovery doesn't depend on the previous block. But the loop blocks on each phase.

The fix: split the work into stages. Each stage processes a range of blocks end-to-end before handing off.

Step 1 — Sketch the stages

let stages = vec![
    HeaderStage,       // download headers for blocks [N..M]
    BodyStage,         // download tx bodies
    SenderRecovery,    // ECDSA-recover senders (parallel)
    Execution,         // run Revm, accumulate state diffs
    Hashing,           // sort hashed account/storage changes
    Merkle,            // compute Merkle roots for the range
    Indexes,           // build txhash → (block, index) etc.
    Finish,            // commit + report
];

for stage in &mut stages {
    stage.run(blocks_n_to_m)?;
}

Now sender recovery batches across blocks, Merkle roots are amortized, and you can parallelize within stages. The data structure is a list of stages, each implementing one trait. Build the trait next.

Step 2 — The first stab at Stage

First attempt:

trait Stage {
    fn execute(&mut self, blocks: BlockRange) -> Result<(), StageError>;
}

One method. Caller passes a range, stage processes it. Done.

This works for forward sync — but it has a critical hole.

Step 3 — unwind: reorgs are not optional

You'd need a separate method, not on this trait — and a separate code path in the orchestrator. Half the codebase becomes "the reorg path." That's exactly what other Ethereum clients have, and exactly what Reth was designed to avoid.

Reth's answer: add unwind to the same trait:

trait Stage {
    fn execute(&mut self, blocks: BlockRange) -> Result<(), StageError>;
    fn unwind(&mut self, blocks: BlockRange) -> Result<(), StageError>;
}

Going forward = call execute over a range. Going back = call unwind over a range. Same trait, two directions. Reorgs become a normal mode of operation, not a special case. This symmetry is the architectural keystone.

Step 4 — ExecInput / ExecOutput: explicit resumability

BlockRange is too thin. The orchestrator needs to tell the stage:

• Where to stop. A target block.
• Where to resume. The stage's checkpoint from the last run (after a node restart).

And the stage needs to tell the orchestrator:

• Where it stopped. New checkpoint.
• Whether it's done. If false, the orchestrator should call again — backpressure control.

pub struct ExecInput {
    pub target: Option<BlockNumber>,
    pub checkpoint: Option<StageCheckpoint>,
}
pub struct ExecOutput {
    pub checkpoint: StageCheckpoint,
    pub done: bool,
}
pub struct UnwindInput {
    pub checkpoint: StageCheckpoint,
    pub unwind_to: BlockNumber,
    pub bad_block: Option<BlockNumber>,
}

Atomic call/return. The orchestrator wants exactly one piece of feedback per turn: "I made progress to checkpoint X; whether you call me again is your decision." A separate has_more() would force the orchestrator into two calls per turn and open a class of bugs where checkpoint and has_more disagree.

Step 5 — Async readiness: poll_execute_ready

A stage that downloads headers can't always execute immediately — it has to wait for network responses. But the orchestrator wants to schedule across stages without blocking on one slow stage.

fn poll_execute_ready(&mut self, _cx: &mut Context<'_>, _input: ExecInput)
    -> Poll<Result<(), StageError>>
{
    Poll::Ready(Ok(()))  // default: always ready
}

A Rust async-style poll method.

What Poll<T> is: every Rust Future has an internal fn poll(...) -> Poll<T> that, on each call, returns either Poll::Ready(value) (done) or Poll::Pending (not yet). When it returns Pending, the runtime sets the stage aside and polls a different one. When the stage is ready it gets woken up by another path, and the runtime polls it again. The mechanism for "wait" without blocking a thread.

Stages that are always ready (most of them) take the default. Stages that wait on I/O override it to return Poll::Pending while their futures are in flight.

The orchestrator polls each stage; if pending, it moves on. No stage blocks the others.

Step 6 — Commit hooks: post_execute_commit / post_unwind_commit

Some stages need to do work after their data is committed to disk. These hooks let stages do that without polluting execute with "are we committed yet?" logic.

fn post_execute_commit(&mut self) -> Result<(), StageError> { Ok(()) }
fn post_unwind_commit(&mut self) -> Result<(), StageError> { Ok(()) }

Default no-op; stages override only when they need it. Concrete examples in Reth:

• ExecutionStage uses post_execute_commit to push block-executed notifications to ExEx subscribers — the commit must finish first because subscribers will read the committed data.
• Pruner stages free old indexes on disk after a checkpoint write succeeds.

Most stages don't override. Opt-in lifecycle, not mandatory plumbing.

Step 7 — #[auto_impl(Box)]: heterogeneous stage list

The orchestrator stores stages in a Vec<Box<dyn Stage<...>>>. That requires Stage to be implemented for Box<S> where S: Stage.

Without the attribute, you'd manually write:

impl<S: Stage<P>> Stage<P> for Box<S> {
    // forward all 6 methods through (**self).method(...)
}

auto_impl is a procedural macro that generates this forwarding. With #[auto_impl(Box)], the orchestrator can hold a list of differently-typed stages and call them all through the same trait object.

What you've built

Every piece earned its keep:

• execute / unwind (Steps 3–4) — symmetry: forward and reorg use the same surface
• ExecInput / ExecOutput (Step 4) — explicit resumability across restarts
• done as a flag (Step 4) — atomic call/return
• poll_execute_ready (Step 5) — async readiness, non-blocking scheduling
• post_*_commit (Step 6) — opt-in lifecycle hooks
• #[auto_impl(Box)] (Step 7) — heterogeneous stage list

The next lesson tours Reth's actual 10-stage pipeline — what each stage does and why the order matters.

Recall before moving on

Without scrolling:

1. Why is unwind on the same trait as execute?
2. What does done: bool enable that has_more() wouldn't?
3. Why does poll_execute_ready exist? Which stages would override it?
4. What does #[auto_impl(Box)] save you from writing?

If any answer is shaky, scroll back. The next lesson is Reth's actual pipeline.

🧭 Where you are now in the stack: you've built the database × concurrency layer's ETL pipeline abstraction — Stage's execute / unwind symmetry, explicit ExecInput / ExecOutput, poll_execute_ready for I/O readiness, auto_impl for Box dispatch. Same shape as Airflow / dbt / Kafka Streams pipelines, applied to chain sync. Next lesson walks the real 10-stage pipeline Reth ships with.

Summary (3 lines)

• 6-step buildup: naive → checkpoint → backward → done flag → generic DB → Send/Sync/'static → 6 methods.
• Each step adds a production requirement. Final trait has 6 methods covering forward + backward + progress tracking.
• Next: Reth's 10 stages, in order.