Lesson 10 — build_payload drains pending fills

Question

build_payload is Reth's block-building hook. Override it to drain pending_fills and emit them as L1 events — every block, fills become first-class L1 records.

Principle (minimum model)

• build_payload hook. Reth's EvmConfig::build_payload(&self, attrs, ...) is called for every block. Default impl produces a standard Ethereum payload.
• Override. Build standard payload first; then drain pending_fills from the LiveRethEvmBridge; emit each fill as a system tx (zero-gas + system address).
• System tx structure. to = bridge_address, data = encode(fill), gas = 0. Reth's payload builder accepts system txs; they're processed at the end of the block.
• Why a system tx. Atomicity with the block. If the block is finalised, fills are too. No separate event queue, no replay risk.
• Idempotency. pending_fills is drained atomically; even if the block fails to finalise, the next block re-attempts.
• Tests. Submit an order that fills; build a payload; assert the payload contains a system tx with the fill encoded.
• Production performance. Drain is O(n) in the number of fills per block. For high-volume markets, batch into one large system tx instead of N small ones.

Worked example + steps

Lesson 10 — build_payload drains pending fills

Goal

Concepts you'll grasp in this lesson:

• std::mem::take is O(1) — a pointer swap, not element copying — for a Vec<Fill> with 1000 entries, mem::take swaps the (ptr, len, cap) triple in one assignment; drain(..).collect() is O(N) with iterator overhead. Knowing the stdlib primitive saves you from inventing slower versions.
• Drain at build_payload, not at submit — fills get grouped by which payload they ride in, not by submission order. If we drained at submit time, the bridge would need a side-channel to track payload assignment. Buffer-then-drain encodes the grouping for free.
• Forward-only drain mirrors block immutability — payload N gets the fills accumulated between the previous build_payload and now. Earlier payloads are not retroactively updated. This is the same semantics as committed blocks: once built, frozen.
• Two short locks beat one long lock when operations are independent — we lock state to compute the payload ID, briefly lock pending_fills for the swap, then continue under state's lock to insert. The pending_fills mutex isn't held during the heavy work.
• The lost-fill failure mode is real but acceptable for v0 — if build_payload errors after the drain, the fills are gone from pending_fills but never made it into a payload. Production hardening would add a recovery queue; v0 single-validator devnet accepts the risk.

Verification:

cargo test -p openhl-evm --release

…still passes 38 tests.

Specific changes:

One small change in build_payload — about 8 lines — replaces Lesson 9's Vec::new() placeholder with std::mem::take(...), so each new payload drains every fill the CLOB has accumulated since the last build_payload call.

The drain is forward-only: once a fill is attached to payload N, it's gone from pending_fills and will not appear in payload N+1. This is the data-flow promise the bridge makes to its consumers — each payload owns exactly one snapshot of fills, taken at build time.

Lesson 10 is short (one focused change). Lesson 11 will write the integration test that exercises the full pipeline.

Recap

After Lesson 9, the bridge has:

// new fields
clob: Mutex<Book>,
pending_fills: Mutex<Vec<Fill>>,

// new methods
pub fn submit_order(&self, order: Order) -> FillResult     // pushes fills
pub fn payload_fills(&self, id: PayloadId) -> Option<Vec<Fill>>  // reads fills
pub fn pending_fill_count(&self) -> usize                  // reads count

You can submit orders. Fills accumulate in pending_fills. pending_fill_count() reports the buffer size. But build_payload ignores the buffer — it inserts Vec::new() as the third element of the pending tuple. So payload_fills(id) returns Some(vec![]) even when the buffer has entries.

Lesson 10 closes that gap.

Plan

One change, in one location. Inside crates/evm/src/live_node.rs's build_payload method, the line:

s.pending.insert(id, (hash, header, Vec::new()));

…becomes:

let drained_fills = std::mem::take(
    &mut *self
        .pending_fills
        .lock()
        .expect("pending_fills mutex poisoned"),
);
s.pending.insert(id, (hash, header, drained_fills));

That's the whole lesson. Eight lines of code. The interesting parts are what std::mem::take does and why we want forward-only drain semantics.

(Answer: drain(..) removes the elements one at a time, returning an iterator. mem::take swaps the entire Vec<Fill> by value — one pointer swap, no per-element work. For a Vec with N fills, drain is O(N) plus iterator overhead; mem::take is O(1) constant-time. mem::take is faster and clearer for "take everything and reset to default.")

Drawing the forward-only drain semantics along the time axis:

time →

   submit_order(o_a)     submit_order(o_b)     build_payload(id=1)     submit_order(o_c)     build_payload(id=2)
        ↓                     ↓                      ↓                       ↓                      ↓
  pending_fills:          pending_fills:        std::mem::take →         pending_fills:         std::mem::take →
   [fill_a]                [fill_a, fill_b]      payload_id=1 owns          [fill_c]              payload_id=2 owns
                                                 [fill_a, fill_b]           (appended to            [fill_c]
                                                 pending_fills is           drained buffer)         pending_fills is
                                                 empty again                                        empty again

       ←  fills going into payload id=1  →   ←  fills going into payload id=2  →

pending HashMap (written by build_payload):
   {                                            {
     id=1: (hash_1, header_1, [fill_a, fill_b])   id=1: (..., [fill_a, fill_b])  ← no retroactive write
                                                   id=2: (hash_2, header_2, [fill_c])
   }                                            }

Forward-only semantics: a payload built at moment T owns exactly the fills that had accumulated up to T. Fills arriving after T never attach retroactively to an earlier payload. Each build_payload call is a cut on the timeline. This is the load-bearing invariant that Lesson 11's integration tests will pin down.

Walk-through

Step 1: Find the line to change

Open crates/evm/src/live_node.rs. Inside impl<P> ConsensusBridge for LiveRethEvmBridge<P>, find build_payload. Scroll to near the end of the body (just before Ok(PayloadId(id))). You should see the Lesson 9 placeholder line:

        let hash = header.hash_slow();
        s.pending.insert(id, (hash, header, Vec::new()));    // empty Vec<Fill> for now; Lesson 10 drains pending_fills here
        Ok(PayloadId(id))
    }

The comment from Lesson 9 explicitly points here. This is the spot.

Step 2: Replace with the drain

Change the section from let hash = header.hash_slow(); through the insert to:

        let hash = header.hash_slow();

        // Drain whatever fills the CLOB has accumulated since the last
        // build_payload call. The fills attach to this payload so the bridge
        // can route them downstream (encode as EVM txs, return via
        // payload_fills, etc.). 8d keeps them as a parallel list; future
        // stages encode them into the block body.
        let drained_fills = std::mem::take(
            &mut *self
                .pending_fills
                .lock()
                .expect("pending_fills mutex poisoned"),
        );

        s.pending.insert(id, (hash, header, drained_fills));
        Ok(PayloadId(id))
    }

Two new statements: the let drained_fills block and the modified insert. The comment is intentional — it documents the drain-on-build semantics for future readers.

Walk the new code carefully:

1. self.pending_fills.lock() — acquire the mutex. Returns LockResult<MutexGuard<Vec<Fill>>>. The .expect("pending_fills mutex poisoned") unwraps the result (expect over poisoned mutexes is fine — see Lesson 9's design reflection).
2. .lock().expect(...) returns a MutexGuard<Vec<Fill>>. MutexGuard is Deref<Target = Vec<Fill>>, but it also has DerefMut. To take ownership of the Vec by value, we need &mut Vec<Fill>, which we get by &mut *guard.
3. std::mem::take(&mut *guard) does the swap: the Vec's heap-pointer + len + capacity move out of the MutexGuard into our drained_fills variable; the MutexGuard's Vec is replaced with Vec::default() (which is Vec::new() — an empty Vec with no allocation).
4. The MutexGuard is dropped at the statement's ; — the temporary MutexGuard produced inside let drained_fills = std::mem::take(&mut *self.pending_fills.lock().expect("...")) ; lives only for the duration of that right-hand side. Rust's scoping rule drops it at the terminating ;, releasing the lock before the next line s.pending.insert(...) runs. This matters for lock ordering: if some other path held state.lock() while reaching for pending_fills.lock() in the opposite order, you'd risk a deadlock — Lesson 10's structure leaves no such window.
5. s.pending.insert(id, (hash, header, drained_fills)) stores the snapshot of fills with the new payload. The pending_fills buffer is now empty, ready for the next round of submits.

The whole std::mem::take(...) expression is a single atomic operation under the lock — no other caller can see "half-drained" state. Either pending_fills is full or it's empty; never mid-drain.

Step 3: Verify nothing else changed

Run cargo check -p openhl-evm. You should see only the line you just modified compile differently — no ripple effects, no other tests broken. The signature of build_payload is unchanged (still async fn ... -> Result<PayloadId, BridgeError>), so callers don't notice.

If you want a quick mental test of "are the fills actually moving?":

// Conceptually:
bridge.submit_order(order1);  // fill F1 → pending_fills: [F1]
bridge.submit_order(order2);  // fill F2 → pending_fills: [F1, F2]
assert_eq!(bridge.pending_fill_count(), 2);

let id1 = bridge.build_payload(...).await.unwrap();
// Now pending_fills is empty (drained into payload id1)
assert_eq!(bridge.pending_fill_count(), 0);
// And the payload has the fills attached
assert_eq!(bridge.payload_fills(id1), Some(vec![F1, F2]));

let id2 = bridge.build_payload(...).await.unwrap();  // empty drain this time
assert_eq!(bridge.payload_fills(id2), Some(vec![]));  // no retroactive fills

This is roughly what Lesson 11's integration test does, but executed against a real Reth node bootstrap. Lesson 10 just makes the underlying mechanism work.

Test

cargo test -p openhl-evm --release

After ~30 seconds (incremental compile):

... 38 tests ...

test result: ok. 38 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

Course 6's existing tests still pass. The Lesson 9 + Lesson 10 changes are structural — the matching engine isn't being exercised by any existing test (those come in Lesson 11), but everything that did work before still works.

You can confirm the change took effect with a quick grep:

grep -n "std::mem::take" crates/evm/src/live_node.rs
# Should report 1 line in build_payload — your new change.

grep -n "Vec::new()" crates/evm/src/live_node.rs
# Should NOT report the line in build_payload anymore. (Other Vec::new() in the
# file, like for the initial pending_fills initialization, are fine.)

Common errors and fixes:

• *error[E0596]: cannot borrow self.pending_fills.lock()... as mutable — the lock returns a LockResult which needs .expect(...) (or .unwrap()) to unwrap to MutexGuard. Re-check that .lock().expect("...") chain.
• **error[E0277]: MutexGuard<'_, Vec<Fill>>doesn't implementDerefMut** — make sure you're using &mut *guardand not&*guard. The *guardderef +&mutborrow is what gives you&mut Vec<Fill>`.
• error: cannot move out of borrowed content — you tried something like std::mem::take(self.pending_fills.lock().expect(...)) (no &mut *). The mem::take signature is fn take<T: Default>(dest: &mut T) -> T. The argument has to be a &mut, and dereferencing the MutexGuard gives you the right shape.

Design reflection

Three load-bearing decisions encoded here:

1. Drain at build_payload, not at submit. Submits push into pending_fills; only build_payload empties it. This is intentional — fills are grouped by the payload they were assembled into, not by the order they came in. The downstream payload-consumer gets a coherent "this batch of fills happened between the previous payload and this one." If we drained at submit time, the bridge would need a side-channel to track which fills go with which payload — more state, more bookkeeping.

2. std::mem::take is the right primitive. It's O(1), atomic under the lock, and signals intent ("take everything, leave default"). The alternative — collect::<Vec<_>>(...drain(..)) then explicit clear — is O(N) + has a half-drained window. Knowing the standard-library primitives saves you from inventing slower or buggier versions.

3. The drain is forward-only. Payload N attaches fills produced between (the previous build_payload call) and (this call). Earlier payloads aren't updated with fills that arrive later. This matches the chain's semantics: once a block is built, its contents are frozen. The buffer-then-drain shape encodes "what's in this block" without requiring an explicit grouping mechanism.

Answer key

cd ~/code/openhl-reference
git checkout 428cc26
diff -u ~/code/my-openhl/crates/evm/src/live_node.rs ./crates/evm/src/live_node.rs

After Lesson 10, the bridge code is functionally equivalent to 428cc26 modulo doc comments. The only difference from the reference should be the integration test — clob_fills_flow_into_payload doesn't exist in your code yet. That's Lesson 11.

Return:

git checkout main

Common questions

Q: What if pending_fills has many fills (say 1000)? std::mem::take is still O(1). The Vec itself owns a heap allocation; mem::take swaps the (pointer, length, capacity) triple. No element-by-element work. The downstream consumer eventually iterates over the 1000 fills, but that's their cost, not the drain's.

Q: Could two build_payload calls race and both think they have the full fill set? No, because std::mem::take is under a MutexGuard. While the lock is held, no other thread can acquire the lock. The first build_payload gets the full set; the second gets an empty Vec (because the first replaced it with Vec::default()). The mutex serializes the drains.

Q: What if build_payload errors out after the drain? The fills are gone from pending_fills but never made it into a payload. They're effectively lost — submitted but not committed. This is a real bug class that production code should handle (e.g., save the drained fills to a recovery queue before doing the rest of build_payload). For our v0 single-validator devnet, the failure path is rare enough that we accept the loss; production hardening is downstream work.

Q: Why is drained_fills not in the state lock — it's locked separately? Because pending_fills and state are separate mutexes (the Lesson 9 design decision). We lock state first (to compute the new payload ID), then briefly lock pending_fills (just for the swap), then continue using the state lock to insert into pending. Two short locks beat one long lock when the operations are independent.

Next lesson (Lesson 11)

The bridge has the data flow. Nothing yet proves it works end-to-end. Lesson 11 writes the clob_fills_flow_into_payload integration test:

1. Bootstrap a real Reth EthereumNode (same pattern as course 6).
2. Construct LiveRethEvmBridge with the live provider.
3. Call build_payload on an empty book — verify no fills attached (payload_fills returns Some(vec![])).
4. Submit a maker BID @ 100, then a crossing taker SELL @ 100 — fill is produced.
5. Verify pending_fill_count == 1.
6. Build the next payload — verify the fill is drained AND attached.
7. Verify pending_fill_count == 0.
8. Verify the earlier (pre-orders) payload was NOT retroactively filled (drain is forward-only).

After Lesson 11, you have a single integration test that exercises the entire Course 7 pipeline. That's the "we built a working CLOB-integrated bridge" milestone.

Summary (3 lines)

• build_payload override drains pending_fills and emits them as zero-gas system txs. Fills become first-class L1 records.
• Atomicity with the block; idempotent drain. System tx address = bridge, data = encoded Fill, gas = 0.
• Tests assert payload contains the fill system tx. Production: batch into one large tx for performance. Next: integration test milestone.