Lesson 9 — LiveRethEvmBridge gets a CLOB + submit_order

Question

LiveRethEvmBridge (the Bridge contract's Rust-side state) gets a clob: Arc<RwLock<Clob>> field and an submit_order API. The integration point where Solidity-driven orders meet the Rust matching engine.

Principle (minimum model)

• LiveRethEvmBridge struct. Adds clob: Arc<RwLock<Clob>> field. Initialised in the factory; shared with precompiles (Precompiles course).
• submit_order signature. fn submit_order(&mut self, account, market, side, size, limit_price, self_trade_policy) -> FillResult.
• Implementation. Acquire write lock; build Order; call clob.submit(order); release lock; return FillResult.
• Atomicity. All work inside the lock; no .await while holding. Block-level consistency guaranteed.
• Pending fills queue. When a fill happens, push to pending_fills: VecDeque<Fill>. Drained by build_payload (next lesson) → fills emitted as L1 events.
• Per-account locking. Production: per-account locks instead of a global lock for concurrency. Out of scope here; the Capstone notes this as deferred.
• Tests. Submit an order via submit_order; assert it appears in the book; assert pending_fills is updated on a match.

Worked example + steps

Lesson 9 — LiveRethEvmBridge gets a CLOB + submit_order

Goal

Concepts you'll grasp in this lesson:

• The CLOB lives next to the bridge, not inside the Reth EVM — clob: Mutex<Book> is a field on LiveRethEvmBridge, alongside provider and state. Fills become a parallel data lane that rides along with each payload; they aren't yet EVM transactions (that's course 8's precompiles). This is the architectural shape of "CLOB on top of EVM."
• Lock granularity: two Mutexes, not one — clob and pending_fills are mutated at different times by different callers. Splitting locks means a thread reading pending_fill_count doesn't block submitters touching the book. Lock granularity matters when contention is on the hot path.
• Interior mutability + &self is the idiomatic shape for async-shared state — submit_order(&self, ...) lets the bridge be wrapped in Arc and shared across tasks without a top-level RwLock<Bridge> that would serialize everything.
• APIs that lock should never hand references back through the lock — payload_fills returns Vec<Fill> (cloned), not &[Fill], because returning a borrow would force the caller to hold the lock guard for the slice's lifetime — instant deadlock with anything else that wants the lock.
• Empty-Vec placeholder is more discoverable than a TODO comment — build_payload inserts Vec::new() until Lesson 10 swaps in std::mem::take(...). Readers see exactly where the missing functionality lives; a comment would rot.

Verification:

cargo test -p openhl-evm --release

…still passes (38 tests from course 6 + 1 new test from Lesson 9). The bridge now owns a CLOB matching engine.

Specific changes:

• One new workspace dep — openhl-clob = { workspace = true } in crates/evm/Cargo.toml.
• Two new fields on LiveRethEvmBridge: clob: Mutex<Book> and pending_fills: Mutex<Vec<Fill>>.
• A wider pending tuple — pending: HashMap<u64, (B256, Header)> becomes HashMap<u64, (B256, Header, Vec<Fill>)>. The third element is the per-payload fill list.
• Three new methods on the bridge: submit_order(&self, order: Order) -> FillResult, payload_fills(id) -> Option<Vec<Fill>> (inspection), pending_fill_count() -> usize (inspection).
• Ripple updates — destructuring on the pending tuple in build_payload, payload_ready, validate_payload, commit all need the 3-tuple pattern.

build_payload doesn't drain pending_fills yet — it inserts an empty Vec<Fill> for now. Lesson 10 makes the drain real. After Lesson 9 you can submit orders, see fills accumulate in pending_fills, but the bridge's payloads carry no fills. Lesson 10 closes that gap; Lesson 11 writes the integration test that proves it.

Recap

After course 6 (consensus, through Lesson 14) + course 7 (CLOB, through Lesson 8), your workspace has:

crates/clob/                            — complete matching engine (Lessons 1–8)
crates/evm/src/live_node.rs             — LiveRethEvmBridge<P>
  fields: provider, chain_spec, validator, engine_handle: Option<...>, state: Mutex<State>
  pending: HashMap<u64, (B256, Header)>
crates/consensus/                       — full BFT engine

cargo test -p openhl-evm passes 38 tests. The CLOB exists, the bridge exists, but they don't know about each other. Lesson 9 wires the bridge to the CLOB.

Plan

Six things, all in crates/evm/:

1. Add openhl-clob = { workspace = true } to crates/evm/Cargo.toml's [dependencies].
2. Add the import to crates/evm/src/live_node.rs: use openhl_clob::{Book, Fill, FillResult, Order};.
3. Add clob + pending_fills fields to the LiveRethEvmBridge<P> struct.
4. Change pending to a 3-tuple in the State struct.
5. Update new() to initialize the new fields.
6. Add three methods to the impl<P> LiveRethEvmBridge<P> block: submit_order, payload_fills, pending_fill_count.
7. Ripple-update the destructuring in build_payload, payload_ready, validate_payload, commit to match the new 3-tuple shape. build_payload inserts an empty Vec<Fill> for now.

Step 7 sounds tedious but is mechanical: every place that wrote (hash, header) or (h, _) becomes (hash, header, fills) or (h, _, _). The compiler tells you each location with a clear error.

The bridge's internal topology after Lesson 9, in one diagram:

        order in
            ↓
   ┌───────────────────────────────────┐
   │  LiveRethEvmBridge<P>             │
   │                                   │
   │   ┌─────────────────┐             │
   │   │ Arc<Mutex<Book>>│ ← submit_order locks briefly to match,
   │   │   (matching)    │   returns the FillResult
   │   └─────────────────┘             │
   │            │ fills                │
   │            ↓                      │
   │   ┌─────────────────────┐         │
   │   │ Mutex<Vec<Fill>>    │ ← submit_order locks briefly to append.
   │   │   (pending_fills)   │   Lesson 10's build_payload drains it.
   │   └─────────────────────┘         │
   │            │                      │
   │            ↓                      │
   │   ┌──────────────────────────┐    │
   │   │ Mutex<State>             │    │
   │   │   pending: HashMap<id,   │    │
   │   │     (hash, header,       │ ← Lesson 10 will inject fills as Vec<Fill>;
   │   │      Vec<Fill>)>         │   today we insert an empty Vec
   │   └──────────────────────────┘    │
   └───────────────────────────────────┘
            │ build_payload → PayloadId
            ↓
        EVM lane (state, header, forkchoice)

The load-bearing decision is keeping clob and pending_fills in separate Mutexes — the two lanes don't serialize each other, so a long hold on one doesn't delay the other. The EVM lane (State.pending HashMap) is the existing bridge plumbing; the CLOB plugs in as a fully parallel lane.

(Answer: Some(vec![]) — the empty fill list. Lesson 9 wires the data flow but build_payload still inserts an empty Vec instead of draining. Lesson 10's "drain on build" change is what turns this into Some(vec![fill_a, fill_b, ...]).)

Walk-through

Step 1: Add the dep to crates/evm/Cargo.toml

Open crates/evm/Cargo.toml. The current [dependencies] section (after course 6) has the various openhl-types, reth-*, alloy-* deps. Add one line:

[dependencies]
openhl-consensus         = { workspace = true }
openhl-types             = { workspace = true }
openhl-clob              = { workspace = true }      # NEW
async-trait              = { workspace = true }
# ... rest unchanged ...

openhl-clob is already declared in the workspace Cargo.toml (you added the path entry in Lesson 1). The [dependencies] entry says "this specific crate uses it."

Step 2: Add the import to live_node.rs

Open crates/evm/src/live_node.rs. The current imports include all the reth-related types. Add this line above the openhl_consensus import:

use alloy_consensus::Header;
use alloy_primitives::{Address, B256};
use alloy_rpc_types_engine::ForkchoiceState;
use async_trait::async_trait;
use openhl_clob::{Book, Fill, FillResult, Order};                     // NEW
use openhl_consensus::bridge::{BridgeError, ConsensusBridge};
use openhl_types::{BlockHash, ExecutedBlock, PayloadAttrs, PayloadId, PayloadStatus};
// ... rest unchanged ...

Four types pulled in: Book (the matching engine), Fill (output), FillResult (the wrapper from Book::submit), Order (the input to submit).

Also update the module-level doc comment to acknowledge the new stage. Find the existing block of //! Stage 7X comments at the top of the file:

//! Stage 7a: parent lookups go through the live node's provider via the
//! `BlockNumReader` trait.
//!
//! Stage 7c: `validate_payload` runs Reth's `EthBeaconConsensus::
//! validate_header_against_parent` against the live parent ...
//!
//! Stage 7d: `commit` now sends a `ForkchoiceUpdated` to Reth's in-process
//! consensus engine ...

…and insert a new Stage 8d block somewhere reasonable (between 7c and 7d is fine):

//! Stage 8d: the bridge now owns a CLOB matching engine. `submit_order` routes
//! orders into the book and accumulates resulting fills in `pending_fills`.
//! `build_payload` drains the pending fills and stores them alongside the
//! synthesized header, so the payload carries real CLOB-generated content.
//! Fills are not yet encoded as EVM transactions executable by Reth's
//! `BlockExecutor` — that's the next stage (or Module 3). 8d proves the
//! wiring exists; encoding is downstream.

This is meta-documentation — when someone reads the file 6 months from now, the staging comments are the map.

Step 3: Add fields to LiveRethEvmBridge

Find the struct definition. Add two fields between validator and state:

#[derive(Debug)]
pub struct LiveRethEvmBridge<P> {
    provider: P,
    chain_spec: Arc<ChainSpec>,
    validator: EthBeaconConsensus<ChainSpec>,
    clob: Mutex<Book>,                                            // NEW
    pending_fills: Mutex<Vec<Fill>>,                              // NEW
    engine_handle: Option<ConsensusEngineHandle<EthEngineTypes>>,
    state: Mutex<State>,
}

Two Mutex-wrapped fields. Why both Mutex?

• clob: Mutex<Book> — the matching engine. Book itself is not thread-safe internally; wrapping in Mutex lets multiple callers submit orders concurrently (the bridge will be shared via Arc<LiveRethEvmBridge> once integrated into the engine app loop).
• pending_fills: Mutex<Vec<Fill>> — the buffer where submit_order pushes fills and build_payload (in Lesson 10) drains them. Separate Mutex from clob because the two mutate at different times: a submit holds clob's lock briefly to do matching, then briefly holds pending_fills's lock to append. A separate lock means two submits don't serialize through the full submit → push chain.

Step 4: Change the pending tuple

Find the State struct definition:

#[derive(Debug, Default)]
struct State {
    next_payload_id: u64,
    pending: HashMap<u64, (B256, Header)>,
    chain: HashMap<B256, Header>,
    head: Option<B256>,
}

Change pending's value type to a 3-tuple, with the third element being Vec<Fill>:

#[derive(Debug, Default)]
struct State {
    next_payload_id: u64,
    /// Pending payloads keyed by `PayloadId.0`. Value is (`block_hash`, `header`,
    /// fills drained from the CLOB at `build_payload` time).
    pending: HashMap<u64, (B256, Header, Vec<Fill>)>,
    chain: HashMap<B256, Header>,
    head: Option<B256>,
}

chain stays as HashMap<B256, Header> because committed blocks don't need to track their fills here — the fills are downstream of commit. (Production code would persist fills somewhere; that's beyond this course.)

The new doc comment is part of the lesson. It explains why the third element exists — the data flow from submit_order → pending_fills → build_payload drains → per-payload Vec<Fill> in pending map.

Step 5: Update new()

The current new() initializes 4 fields. After the changes, it initializes 6. Update:

impl<P> LiveRethEvmBridge<P> {
    #[must_use]
    pub fn new(provider: P, chain_spec: Arc<ChainSpec>) -> Self {
        let validator = EthBeaconConsensus::new(Arc::clone(&chain_spec));
        Self {
            provider,
            chain_spec,
            validator,
            clob: Mutex::new(Book::new()),                        // NEW
            pending_fills: Mutex::new(Vec::new()),                // NEW
            engine_handle: None,
            state: Mutex::new(State::default()),
        }
    }

Two new field initializations. Book::new() from Lesson 3's helper (workspaces are wired so openhl_clob::Book::new() is callable here). Vec::new() for the empty fill buffer.

Step 6: Add the three new methods

Below new() (or after chain_spec() if you prefer grouping pub methods together), add:

    /// Submit an order to the CLOB. Resulting fills are buffered in
    /// `pending_fills` until the next `build_payload` drains them.
    pub fn submit_order(&self, order: Order) -> FillResult {
        let mut book = self.clob.lock().expect("clob mutex poisoned");
        let result = book.submit(order);
        if !result.fills.is_empty() {
            self.pending_fills
                .lock()
                .expect("pending_fills mutex poisoned")
                .extend(result.fills.iter().copied());
        }
        result
    }

    /// Inspect (read-only) the fills attached to a built payload. Returns
    /// `None` if the payload id is unknown. Production code would encode
    /// these as EVM-executable transactions before they reach the block
    /// body; v0 keeps them as a parallel list for test inspection.
    #[must_use]
    pub fn payload_fills(&self, id: PayloadId) -> Option<Vec<Fill>> {
        let s = self.state.lock().expect("state mutex poisoned");
        s.pending.get(&id.0).map(|(_, _, fills)| fills.clone())
    }

    /// Number of fills currently buffered, waiting for the next `build_payload`.
    #[must_use]
    pub fn pending_fill_count(&self) -> usize {
        self.pending_fills
            .lock()
            .expect("pending_fills mutex poisoned")
            .len()
    }

Three methods, three intents:

• submit_order — the write path. Takes &self (not &mut self) because internal mutability via Mutex lets shared references mutate the bridge. Locks clob, calls book.submit, gets back a FillResult. If any fills were produced, locks pending_fills and appends them. Returns the FillResult so the caller knows what happened.

  Lock-ordering safety — important: the source makes it look like let mut book = self.clob.lock()... stays alive into the middle of the function, but Rust's non-lexical lifetimes (NLL) drop book (the MutexGuard) immediately after book.submit(order) (its last use). By the time pending_fills.lock() runs, the clob lock is already released. The two locks are never held simultaneously — they're acquired and released in series. No deadlock path exists; the compiler enforces the drop timing.

  For maximum explicitness, you could wrap the first step in a scope block — let result = { let mut book = ...; book.submit(order) }; — or insert drop(book); right after book.submit. This course keeps the byte-identical form against the openhl reference SHA, but in production code the explicit scope or drop is a defensive habit that makes it harder to accidentally introduce "hold one lock while taking another" in a later refactor.

• payload_fills — the inspection path. Returns Option<Vec<Fill>> for a given PayloadId. None if the id isn't in pending; Some(vec) (possibly empty) if it is. The doc comment is explicit that this is a test-and-debug method — production code would route fills through a transaction-encoding pipeline.

• pending_fill_count — a small debugging helper. How many fills are sitting in the buffer waiting to be drained. Useful for tests like "submit two orders that cross, expect count == 1."

Notice all three methods take &self. The internal Mutexes do the heavy lifting; the public API is "shared reference + interior mutability," which is exactly what async code needs (multiple async tasks can hold &LiveRethEvmBridge concurrently).

Step 7: Ripple-update the destructuring

This is the tedious-but-mechanical part. The pending tuple is now 3 elements; every place that pattern-matches on it needs to know. Five sites total:

Site 1: build_payload — search for s.pending.insert(id, ...). Currently:

let hash = header.hash_slow();
s.pending.insert(id, (hash, header));
Ok(PayloadId(id))

Change to:

let hash = header.hash_slow();
s.pending.insert(id, (hash, header, Vec::new()));    // empty Vec<Fill> for now; Lesson 10 drains pending_fills here
Ok(PayloadId(id))

Vec::new() is the placeholder. Lesson 10 replaces it with std::mem::take(&mut *self.pending_fills.lock()...).

Site 2: payload_ready — search for s.pending.get(&n).cloned(). Currently:

let (hash, header) = s
    .pending
    .get(&n)
    .cloned()
    .ok_or_else(|| BridgeError::Rejected(format!("unknown payload id {n}")))?;

Update the destructuring:

let (hash, header, _fills) = s
    .pending
    .get(&n)
    .cloned()
    .ok_or_else(|| BridgeError::Rejected(format!("unknown payload id {n}")))?;

The _fills binding catches the new third element but doesn't use it — payload_ready returns an ExecutedBlock, which doesn't need the fills directly. The _ prefix tells the compiler "we know it's there, we don't need it."

Site 3: validate_payload — inside the let header = { ... } block, search for .find(|(h, _)| *h == block_hash):

.find(|(h, _)| *h == block_hash)
.map(|(_, h)| h.clone())

Update both closures to 3-element patterns:

.find(|(h, _, _)| *h == block_hash)
.map(|(_, h, _)| h.clone())

Site 4: commit — search for the same .find(|(h, _)| *h == hash) pattern, change identically:

let header = s
    .pending
    .values()
    .find(|(h, _, _)| *h == hash)
    .map(|(_, h, _)| h.clone())
    .ok_or_else(|| ...)?;

Site 5: payload_fills (the new method you just added in Step 6) — already uses the 3-element pattern in the .map(|(_, _, fills)| fills.clone()) line. No change needed.

That's all 5 sites. Run cargo check -p openhl-evm — if you missed any, the compiler will tell you with a "pattern matches against tuple of length 2 but expected 3" error.

Test

cargo test -p openhl-evm --release

After ~30 seconds (incremental compile + node bootstrap):

... 38 tests ...

test result: ok. 38 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

All course 6 tests still pass. Lesson 9 doesn't add new tests — the new functionality (submit_order, etc.) gets exercised by Lesson 11's integration test. The Lesson 9 change is structural — the bridge now has new fields and methods, but the existing test surface doesn't touch them, so all those tests continue to work.

You can do a quick sanity check that the new methods are wired correctly:

// In your existing live_bridge_builds_on_real_genesis test or a new smoke test:
let bridge = LiveRethEvmBridge::new(handle.node.provider.clone(), chain_spec);
assert_eq!(bridge.pending_fill_count(), 0); // empty after fresh bridge

That should pass. We're not testing the matching path yet (that's Lesson 11) — just that the new method compiles and returns 0 for a fresh bridge.

Common errors and fixes:

• error[E0432]: unresolved import 'openhl_clob' — Cargo.toml is missing the dep. Re-check Step 1.
• error[E0277]: 'Mutex<Book>' is not 'Send' — somewhere a Book is being held across an .await. Check that submit_order and pending_fill_count finish their lock + work before any await (they should — they're all synchronous in their bodies).
• error: pattern requires 2 fields, struct has 3 — you missed a ripple-update site. The compiler will name the file:line. Add the third pattern element (_fills or _).
• error: cannot find value 'pending_fills' in build_payload — you didn't add the field to the struct or to new(). Re-check Steps 3 and 5.

Design reflection

Three load-bearing decisions encoded here:

1. Two Mutexes instead of one. The bridge's CLOB state and its fill buffer are different concerns mutated at different times. Splitting locks lets concurrent submits avoid blocking each other unnecessarily. Lock granularity matters when contention is on the hot path.

2. submit_order takes &self. Interior mutability via Mutex lets shared references mutate the bridge. The bridge will be wrapped in Arc and shared across async tasks; making methods take &mut self would require RwLock<Bridge> at the top, which would serialize all access through one global lock. Internal Mutex + &self API is the idiomatic Rust pattern for async-shared state.

3. Empty Vec<Fill> placeholder in build_payload. Lesson 9 wires the structure; Lesson 10 makes it functional. Leaving the placeholder is honest scoping — readers can see exactly where the missing functionality lives. A Vec::new() placeholder is more discoverable than a future TODO comment.

Answer key

cd ~/code/openhl-reference
git checkout 428cc26
diff -u ~/code/my-openhl/crates/evm/src/live_node.rs ./crates/evm/src/live_node.rs
diff -u ~/code/my-openhl/crates/evm/Cargo.toml ./crates/evm/Cargo.toml

After Lesson 9, your code is partway through 428cc26's full set of changes — fields + methods are in, but build_payload doesn't yet drain (Lesson 10) and there's no integration test (Lesson 11). The diff should show:

• ✅ clob + pending_fills fields (matches reference)
• ✅ submit_order, payload_fills, pending_fill_count methods (matches reference)
• ✅ 3-tuple in pending (matches reference)
• ❌ build_payload still inserts Vec::new() — reference uses std::mem::take(...)
• ❌ No clob_fills_flow_into_payload integration test — reference has it

The ❌ items land in Lesson 10 + Lesson 11.

Return:

git checkout main

Common questions

Q: Why does submit_order lock clob, finish, then separately lock pending_fills, instead of holding both locks at once? Because the pending_fills append depends on the result of the matching, not on the matching's intermediate state. After book.submit(order) returns, the FillResult is owned data — we can release clob's lock and safely process the result. Holding both locks would serialize unrelated pending_fills operations (e.g., another caller reading pending_fill_count would block) for no correctness benefit.

Q: Why is payload_fills returning Vec<Fill> (cloned) instead of &[Fill] (borrowed)? Because returning &[Fill] would require the caller to hold the state Mutex's lock guard for the lifetime of the slice — which would deadlock anything else that wants the lock. Cloning the Vec is one allocation per payload_fills call, which is fine for an inspection method called rarely. APIs that lock should never return references back through the lock.

Q: Could the clob field be Arc<Mutex<Book>> instead of Mutex<Book>? Yes — and openhl's Stage 9 (later) actually does this, because the CLOB needs to be shared with custom EVM precompiles that read its state. For Stage 8d, plain Mutex<Book> is enough. The change from Mutex<T> to Arc<Mutex<T>> is mechanical — wrap one place, change a few .lock() sites to .lock().expect(...)-on-arc. Defer the Arc wrap until you actually need the sharing.

Q: What happens if pending_fills.lock() panics because of a poisoned mutex? A panic propagates up through submit_order and crashes whatever task called it. In Rust, mutex poisoning happens when a thread panics while holding the lock. In a synchronous body like book.submit(order), panics are rare (the only sources are explicit unwrap()s, OOM, or stack overflow). If they do happen, the bridge is in an inconsistent state anyway — propagating the panic is the right behavior. The .expect("mutex poisoned") is a tripwire, not a recovery path.

Next lesson (Lesson 10)

The bridge has a CLOB and fills accumulate. Payloads built via build_payload still don't carry those fills — the placeholder Vec::new() is the gap. Lesson 10 replaces the placeholder with std::mem::take(&mut *pending_fills.lock(...)) so each new payload drains all accumulated fills. After Lesson 10, bridge.payload_fills(id) returns the actual fills produced since the last build, and bridge.pending_fill_count() resets to 0. Lesson 11 writes the end-to-end test that proves this drain semantic is forward-only (earlier payloads aren't retroactively filled).

Summary (3 lines)

• LiveRethEvmBridge adds clob: Arc<RwLock<Clob>> + submit_order. Acquires lock; calls Clob::submit; returns FillResult.
• Pending fills queue drained by build_payload (next lesson). Atomicity guaranteed within the lock.
• Production: per-account locking for concurrency (deferred). Tests assert submission + pending_fills updates. Next: build_payload drains pending fills.