Lesson 15 — What you built, what's still stub, where to go next

Question

A recap of what the course built (a Malachite engine loop plus a ConsensusBridge backed by a live Reth node), what is still a stub, and where to go next.

Principle (minimum model)

• What you built. A workspace pinned to Reth v2.2.0 and Malachite v0.5.0; shared contract types; the ConsensusBridge trait; InMemoryEvmBridge and alloy-based RethEvmBridge; OpenHlContext with its 10 associated types; Ed25519 OpenHlSigningProvider; OpenHlCodec; OpenHlNode; run_engine_app; a Reth dev-node bootstrap test; and LiveRethEvmBridge. Lessons 0–15.
• What runs end-to-end. Two separate proofs. run_engine_app decides a block through the real Malachite actors with a StubBridge (Lesson 10). LiveRethEvmBridge builds on a live Reth genesis, validates with EthBeaconConsensus and sends forkchoice (Lessons 12–14), tested directly. The course never runs the engine loop against the live bridge.
• What is still stub. No newPayload, so Reth answers SYNCING and its head stays at genesis; blocks carry no transactions; 7 of 8 codec impls return errors; multi-validator gossip is untested; no slashing; no custom OpenHL behaviour (CLOB, precompiles).
• Hyperliquid-shaped, by analogy. Like Hyperliquid, OpenHL pairs a BFT consensus with an EVM. Hyperliquid's node is closed source and not Reth-based, so this is an analogy at the architecture level, not a description of Hyperliquid's code.
• Where the reference went next. In psyto/openhl (now archived): CLOB matching engine (Stage 8a), funding (8b), EVM precompiles (Stage 9), liquidation and ADL (Stage 10), oracle (11), vault (12), a runnable bin/openhl with a Reth-backed devnet (Stage 13), and real Reth payloads with newPayload (Stage 20).

Worked example + steps

Lesson 15 — What you built, what's still stub, where to go next

The system you built

Over 16 lessons you went from cargo init on an empty directory to a single-validator BFT engine that decides blocks through the real Malachite actors (~0.02 s with a stub bridge), plus a bridge that builds, validates and commits against a live Reth node. Your workspace now looks like this:

~/code/my-openhl/
├── Cargo.toml                          ← 17 reth-* deps, 9 malachite deps; git deps SHA-pinned
├── bin/openhl/                         ← (stub binary — production wiring is openhl Stage 13, outside these courses)
├── crates/
│   ├── types/                          Lesson 2:  shared CL↔EL contract types
│   │   └── src/lib.rs                  BlockHash, PayloadId, PayloadAttrs,
│   │                                   ExecutedBlock, PayloadStatus
│   ├── evm/                            EL side (test double → live Reth)
│   │   ├── src/in_memory.rs            Lesson 4:  InMemoryEvmBridge (HashMap state)
│   │   ├── src/engine.rs               Lesson 5:  RethEvmBridge (alloy types, real hash_slow)
│   │   ├── src/reth_node.rs            Lesson 11: bootstrap proof (test-only)
│   │   └── src/live_node.rs            Lessons 12–14: LiveRethEvmBridge<P>
│   │                                   - Lesson 12: parent lookup via BlockNumReader
│   │                                   - Lesson 13: EthBeaconConsensus validate + real header fields
│   │                                   - Lesson 14: ConsensusEngineHandle forkchoice
│   └── consensus/                      CL side (full BFT engine)
│       ├── src/bridge.rs               Lesson 3:  ConsensusBridge trait
│       ├── src/types/                  Lesson 6:  the OpenHl* types behind Context's 10 associated types
│       ├── src/context.rs              Lesson 6:  Context<OpenHlContext> impl
│       ├── src/signing.rs              Lesson 7:  canonical encoding for vote/proposal
│       ├── src/signing_provider.rs     Lesson 7:  SigningProvider<OpenHlContext>
│       ├── src/codec.rs                Lesson 8:  OpenHlCodec (1 real + 7 stub Codec impls)
│       ├── src/node.rs                 Lesson 9:  OpenHlNode + start_engine
│       └── src/engine_app.rs           Lesson 10: run_engine_app (AppMsg routing)

About 30 .rs files in the reference's corresponding crates. Workspace tests: 37 passing (4 types + 21 consensus + 12 evm).

Drawing the full CL ↔ EL integration you opened across this course in one picture makes the boundary you stitched together immediately legible:

   [ CL: openhl-consensus ]                          [ EL: openhl-evm ]
  ┌──────────────────────────────────────────┐    ┌──────────────────────────────────────────┐
  │  Malachite BFT engine (actor system)      │    │   LiveRethEvmBridge<P>                    │
  │                                            │    │                                            │
  │   ├── OpenHlContext                         │    │    ├── provider: P (BlockNumReader        │
  │   │   (10 associated types — Lesson 6)      │    │    │             + HeaderProvider)         │
  │   ├── OpenHlSigningProvider                 │    │    ├── chain_spec: Arc<ChainSpec>          │
  │   │   (Ed25519 + canonical encoding — Lesson 7)│  │    │   (shared source of truth — Lesson 13)│
  │   ├── OpenHlCodec                           │    │    ├── validator:                          │
  │   │   (1 real + 7 stub — Lesson 8)          │    │    │   EthBeaconConsensus<ChainSpec> (Lesson 13)│
  │   ├── OpenHlNode / OpenHlNodeHandle (Lesson 9)│  │    ├── engine_handle:                      │
  │   └── run_engine_app loop                   │    │    │   Option<ConsensusEngineHandle> (Lesson 14)│
  │       (12 AppMsg arms — Lesson 10)          │    │    └── state: Mutex<{ pending, chain,      │
  │                                              │    │                       head, … }>          │
  └──────────────────┬──────────────────────────┘    └──────────────────┬──────────────────────┘
                     │                                                  ▲
                     │ ── all chatter goes through 4 ConsensusBridge ───┘
                     │   methods (the trait surface defined in Lesson 3)
                     │
                     ├── ① build_payload(parent, attrs)
                     │     CL ──► EL : "Assemble the next block."
                     │     EL ──► CL : PayloadId (the bridge synthesises the header itself; no Reth payload builder)
                     │     Under the hood: pull parent_header from provider →
                     │                     ChainSpec::next_block_base_fee + gas_limit copy
                     │                     + timestamp monotonicity → header synth → stash in pending
                     │
                     ├── ② payload_ready(id)
                     │     CL ──► EL : "Hand me the block for that PayloadId."
                     │     EL ──► CL : ExecutedBlock (retrieved from pending)
                     │     ※ The only seam where data flows EL → CL among the four methods
                     │
                     ├── ③ validate_payload(&block)
                     │     CL ──► EL : "Is this block valid?" (called from the tests; run_engine_app doesn't call it yet)
                     │     EL ──► CL : PayloadStatus::Valid / Invalid
                     │     Under the hood: EthBeaconConsensus::validate_header_against_parent
                     │                     (4 sub-checks: number / timestamp / gas-limit / EIP-1559)
                     │
                     └── ④ commit(hash)
                           CL ──► EL : "Quorum reached; finalize this."
                           EL ──► CL : Ok(())
                           Phase 1 (must succeed): state.chain.insert + update head
                           Phase 2 (best-effort):  ConsensusEngineHandle::fork_choice_updated
                               → Reth's in-process Engine API (no body yet → SYNCING reply; discarded)

Three things this picture pins down: (a) The two worlds on either side talk through exactly the four ConsensusBridge methods defined in Lesson 3 — the entire seam between two huge infrastructure stacks fits into that one trait surface. (b) Because run_engine_app (Lesson 10) is generic over B: ConsensusBridge, the same loop accepts any of the four bridge implementations — StubBridge / InMemoryEvmBridge / RethEvmBridge / LiveRethEvmBridge (this course only runs it with StubBridge). That's the polymorphism payoff. (c) The chain_spec: Arc<ChainSpec> inside LiveRethEvmBridge is the shared source of truth referenced by both build_payload and validate_payload — split that, and self-forks appear the moment a hard fork shifts the base-fee formula. Every L1-architect design decision in this course lives somewhere on this single diagram.

The four ConsensusBridge methods — where each ended up

Each row is the closing state of a method after the course:

The bridge talks to Reth's storage layer (HeaderProvider), Reth's chain config (ChainSpec), Reth's consensus validator (EthBeaconConsensus), and Reth's engine actor (ConsensusEngineHandle). What a real CL client also uses, and this course doesn't, is newPayload / getPayload: handing Reth whole blocks and asking Reth to build them.

What's still placeholder

This course shipped a working single-validator chain. It's honest to call out what's not yet there. Each item below is a deliberate scope cut, not an accident:

1. Engine newPayload integration

Status: missing.

commit sends ForkchoiceUpdated, and Reth's engine responds SYNCING because it doesn't have the matching block body. To progress to VALID, you'd need to:

• Encode build_payload's output as a real ExecutionPayload (with a transaction list, even if empty).
• Send it via handle.new_payload(payload).await before the fork_choice_updated call.
• Match the response chain: newPayload → VALID → forkchoice → VALID → canonical head advances.

The bridge builds headers itself, so there is no real ExecutionPayload to send. In the reference repo this was closed later: Stage 20a/20b made build_payload ask Reth's real PayloadBuilder for a block and made commit send engine.new_payload(...) before fork_choice_updated; Stage 20c wired that into bin/openhl and shipped the ExecutionData to followers. OpenHL's own trading state (CLOB fills, accounts) is kept by the bridge and exposed to the EVM through precompiles (Stages 9 and 17), not as user-signed transactions. How Hyperliquid itself moves data between HyperCore and HyperEVM is not public at the code level, so don't read this as a description of Hyperliquid.

2. Real Codec impls

Status: 1 real (OpenHlProposalPart — empty bytes), 7 stubs (return CodecStub error).

In single-validator mode, the codecs for gossiped messages (SignedConsensusMsg, LivenessMsg, StreamMessage), WAL writes (ProposedValue), and peer sync (Status, Request, Response) never fire. Once you add a second validator, every cross-validator message hits one of these stubs.

To extend: pick a wire format (protobuf, borsh, JSON) and write the encode/decode for each type. Malachite's code/crates/test/src/codec/ has two worked examples: a protobuf codec (proto/mod.rs, ~680 lines at v0.5.0) and a JSON one (json/). The reference repo implemented its own OpenHlCodec in Stage 13m.

3. Multi-validator gossip

Status: never exercised.

OpenHlNode already configures libp2p (/ip4/127.0.0.1/tcp/0). What's untested:

• Two OpenHlNode instances discovering each other.
• Vote propagation under network partition.
• Vote-extension exchange.
• Sync of a lagging validator.

Once Codec stubs (#2) are real and you have N=2 nodes spinning up against a shared chain spec, a multi-validator integration test is the natural next step. (The reference repo got there in Stages 13k–13n and 18a, verified with N=3.)

4. Persistent WAL

Status: ephemeral tempdir.

Every test uses temp directories (tempfile::tempdir() for Malachite's home dir, testing_node for Reth's MDBX), so state is gone after each run. OpenHlNode::new already takes a home_dir, so pointing it at a persistent path is mechanical, but verifying crash recovery (kill the node mid-commit, restart, assert the chain head is right) needs real WAL codec impls and a Test Plan that's specifically chaos-engineering shaped.

5. Slashing + double-sign detection

Status: none.

Production BFT chains track validator misbehaviour (signing two different blocks at the same height, voting twice in the same round). Malachite's vote keeper and proposal keeper record equivocation evidence (EvidenceMap); OpenHL doesn't act on it. Building a multi-validator chain without slashing is fine for testnets, dangerous for value-handling networks.

6. Custom Hyperliquid-shape behaviour

Status: vanilla Ethereum.

The whole point of an "openhl-shape" chain is the CLOB and precompiles that make it more than a generic EVM chain. Stage 8 (CLOB matching engine, fills-into-payload) and Stage 9 (custom precompiles, clob_place_order write path) live in psyto/openhl but aren't covered here; the openhl CLOB and precompiles courses build them.

Production-readiness checklist

Working from "I have a passing test" to "I'd let this take real value":

• All 7 Codec stubs replaced with real protobuf/borsh/JSON impls.
• engine_newPayload integration so the engine matches the bridge's view of canonical chain.
• Multi-validator integration test passing with N=2+ nodes against a shared chainspec.
• WAL crash-recovery test (kill mid-commit, restart, verify chain head).
• Persistent home_dir (not tempdir) configured for production deployments.
• Engine SYNCING/VALID/INVALID responses logged with tracing::warn / structured fields, not discarded.
• Slashing/double-sign hooks wired and unit-tested.
• Key rotation procedure (Ed25519 key swap during a chain restart, not at runtime).
• Operational telemetry: Prometheus metrics for round duration, payload build latency, validate failures.
• Performance baseline: blocks-per-second under continuous load (not just smoke test).
• Independent security review of the canonical encoding format (the Lesson 7 byte layout is part of your wire spec).
• Threat model for proposer manipulation under partial network partition.

If you're forking this course's code into a production chain, treat this list as the long-pole work — most of it is harder than the course itself.

What you can now do that you couldn't 16 lessons ago

• Put a Rust BFT engine and a real EL in one workspace. Not "with an FFI to Go" — Malachite's actors and a live EthereumNode both run from the same Rust workspace, joined by one trait.
• Reason about producer/validator self-consistency. When you have a builder and a validator for the same artifact, they must share a source of truth. You've seen this pattern in chain_spec.next_block_base_fee driving both build_payload and validate_payload.
• Apply the incremental-stub pattern. Trait bounds force surface area; if you can't fill it all at once, stub with a clear failure mode. Lesson 8's CodecStub("SignedConsensusMsg<OpenHlContext>") is the model.
• Wire two pieces of generic infrastructure together. Reth and Malachite were written by different teams with different sensibilities. The handshake interface (Node trait, ConsensusBridge trait) is what made them composable. Future courses will use the same pattern with other infra.
• Distinguish protocol errors from operational errors. BridgeError::Rejected vs BridgeError::Internal. PayloadStatus::Invalid vs propagating up. The conversational level matters.
• Write tests that prove the live read happened. Lesson 12's assert_eq!(block.number, 1) was the load-bearing check — anything else would have let an in-memory fallback slip past.

Where to go next

Within these courses:

• The openhl CLOB course — the next step in this build: a matching engine on top of the substrate you just built, followed by the openhl precompiles, funding, liquidation, and ADL courses.
• The expert-level Reth course — deep dives on BlockExecutor, state-root verification, MDBX internals. Natural next once you want validate_payload to actually execute transactions.
• The lesson on running a Reth fork in production — once your devnet runs, this is the immediate next step. Same shape (build flags + monitoring + diff testing + upgrade discipline), applied to a fork that ships.
• The Reth consensus-engineering course — covers slashing, vote extensions, fault tolerance at depth. Where you'd go after multi-validator gossip is working.

Outside these courses:

• psyto/openhl Stages 11 onward — oracle, vault, the runnable devnet binary, multi-validator BFT, the per-block integration tick, and the Stage 20 newPayload work. Source code is in the public repo (archived; its README points active development to psyto/rdk); these stages have no walkthrough course.
• Malachite spec docs (informalsystems/malachite) — read the core-types crate's docs straight through. Half of it is now familiar; the other half is what multi-validator requires.
• A real Reth dev node — clone paradigmxyz/reth, run cargo run --bin reth -- node --dev. Lesson 11's test launched the same EthereumNode with the same --dev setting, from code instead of the CLI. Compare the surface.
• category-labs/monad-bft — a second Rust BFT consensus implementation, actively developed (GPL-3.0). Where Malachite treats consensus as a generic state-machine library with a context type the embedding chain plugs into, Monad-BFT is purpose-built for Monad's own execution layer, which runs asynchronously behind consensus. The two represent opposite honest trade-offs: Malachite optimizes for embeddability (easy to wire into anything, which is exactly what Lessons 6–10 of this course did); Monad-BFT optimizes for single-chain throughput (faster, but harder to reuse). Worth reading after this course to internalize that "BFT in Rust" isn't a single shape. License note: GPLv3 means citing or studying it is fine; never copy code into your openhl tree — openhl is permissive-licensed and would inherit the copyleft.

Closing note

The corresponding files in the reference come to roughly 2,500 lines of Rust including tests. That code is a working single-validator BFT engine plus a Reth-backed bridge, the skeleton of a Hyperliquid-shaped L1. It's not production-ready; it doesn't need to be. What you have is a foundation that's honest about its scope, has every load-bearing decision visible, and is one extensible interface away from each next capability.

The hardest part of an L1 isn't writing the engine — Malachite did most of it, and we just wired it. The hardest part is being honest about what your code can and can't do, and writing tests that prove the can side. Wherever there was a way to fail, the live-bridge lessons paired the happy-path assertion with a negative-path one. That's the discipline that takes you from "the test passes" to "the system works."

Now go build something that uses this.

Summary (3 lines)

• You built: workspace + contract types + ConsensusBridge + in-memory/alloy bridges + Malachite Context, signing, codec, node + run_engine_app + Reth bootstrap + LiveRethEvmBridge (provider read, EthBeaconConsensus, forkchoice). 16 lessons.
• Still stub: no newPayload (Reth answers SYNCING), no transactions, 7 codec stubs, untested multi-validator gossip, no slashing. Analogy only: a Hyperliquid-shaped BFT + EVM design (Hyperliquid's own node is closed source).
• The reference repo went on to the CLOB (Stage 8), precompiles (9), a runnable devnet binary (13) and real Reth payloads (20).